Government 
pubieaiess 
CH RON 
OTF 
eatT 
M3l y.3 


Government of 
Ontario 


Ministry of 
Community and 
Social Services 


Business 
Transformation 
Project Review 


Volume 3, 
Appendices 


HLB Project No. 6573 


February, 1999 


Digitized by the Internet Archive 
in 2022 with funding trom 
University of Toronto 


https://archive.org/details/31/61114708449 


Government of 
Ontario 


Ministry of 
Community and 
Social Services 


Business 
Transformation 
Project Review 


Volume 3, 
Appendices 


HLB Project No. 6573 


February, 1999 





GOVERNMENT OF ONTARIO 


Ministry of Community and Socials Services 


Business Transformation Project Review 


Volume 3, 


Appendices 


Prepared by 


HICKLING LEwIS BROD INC. 


HLB Reference No. 6573 


February, 1999 


Hickling Lewis Brod Inc Page 1! 








& srnvulaV 


,, aesibaneqqak 


“a borane 


wil coat awit LOI 


EV23.0K4 ianersted AItt 
O20! ,qrauaist 





Appendix A_ Review of Current Canadian Government Direction in Procurement 
Appendix B_ Review of Current US Direction in Procurement 
Appendix C Business Case Methodology, Analysis and Results 


Appendix D Business Case Development Guide for Common Purpose Procurement 


Hickling Lewis Brod Inc Page Ill 





APPENDIX A 


REVIEW OF CURRENT CANADIAN GOVERNMENT 


DIRECTION IN PROCUREMENT 


BACKGROUND 


As the Canadian government faces increased pressure to streamline its activities in an effort to 
control its budget, departments are looking to information technology (IT)-based solutions. IT- 
based solutions are looked to by federal government departments as a means by which programs 
and services may be delivered more efficiently and effectively. 


While in some cases, investment in such solutions have produced great dividends, in others, 
expectations have not been met. Problems arise either because projects can not be completed 
within budget or because they do not, when completed, deliver the desired results. 


Though not the only factor affecting project success, numerous reports have shown that the 
procurement process itself is one factor contributing to failure. As a result, Public Works and 
Government Services Canada (PWGSC), the federal government’s procurement and contracting 
authority, in an action plan endorsed by the Treasury Board, called for the development of a new 
method of procurement. 


To address these concerns and requirements, Benefits-Driven Procurement (BDP) has been 
developed by PWGSC. 


BDP Principles and Objectives 


Essentially, BDP is “a value-added, efficient procurement strategy and methodology for the 
acquisition and implementation of IT-based business solutions that will facilitate the 
achievement of client-defined benefits and, in so doing, contribute to the success of such 
projects.” 


BDP is an open competitive process used to select vendors that will possess contractual rights to 
complete all phases of IT-based investment projects. These rights are revocable and are normally 
reviewed at all project milestones. Renewal of these rights are conditional, not solely on delivery 
of a product or service, but rather the delivery of value. 


BDP is an alternative to traditional methods of procurement which emphasize compliance and 
lowest cost rather than best value. By examining expected costs, benefits, and risks of such 


bene ee a st 
Hickling Lewis Brod Inc Page A-1 


investment projects, BDP measures the expected value to be received by the government from 
each investment option. 


The objectives of BDP, as defined by PWGSC, are: 


e To develop and promote a timely competitive procurement process for government and 
industry; 


e To increase the success of the government’s IT-based solutions and, consequently, the value 
for money attained; 


e To contribute to the resolution of procurement process problems and to facilitate the 
implementation of the principles contained in Treasury Board’s An Enhanced Framework for 
the Management of Information Technology Projects. 


BDP is based upon certain principles that provide broad parameters within which a disciplined 
yet flexible procurement alternative to the current method may be established. PWGSC has 
identified three sets of principles: Core Principles, Defining Principles and Sustaining Principles. 
They are described below: 


e Core Principles: These principles are based on the often-complementary roles and 
relationships of client departments, PWGSC contracting officials, and vendors. These 
principles stipulate that value for money, integrity, fairness, prudence and probity are 
essential in the procurement process. In addition, these principles require that the 
relationships of all parties be based on cooperation, where risks are identified and 
appropriately assigned to each party to the procurement process. 


e Defining Principles: These principles are the same as those defined in An Enhanced 
Framework for the Management of Information Technology Projects. These principles are: 
Projects are aligned with, and support, business directions and priorities; Clear 
accountabilities are established; Project managers are developed and work within a corporate 
discipline; and Project management decisions are based on risk management which considers 
both the risks and their mitigation throughout the life-cycle of the project. 


e Sustaining Principles: These principles deal with how BDP is performed. These principles 
are: Flexible procurement strategy and process that is driven by the outcomes and benefits of 
the project; Sound continuous risk management; Tight integration of the procurement 
activities with the project management process; Increased visibility of the cost element 
throughout the project; and Constant focus on process and organizational capability. 


BDP Process 
As defined by PWGSC, the BDP process is as follows: 


1. The client department prepares its Business Plan and Main Estimates for expenditures in 
accordance with Treasury Board requirements for Program Forecasts. 
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Once approved, the department develops a Business Case for each IT-based solution 
investment alternative it wishes to consider (in accordance with its Business Plan and Main 
Estimates). An independent third party performs a capability evaluation of the client 
department and a front-end risk assessment of the investment project. 


For each project, the Business Case, capability evaluation, and front-end risk assessment is 
discussed with PWGSC and a recommended procurement strategy consistent with the scope, 
complexity and other characteristics of the project is identified. If the project typifies the 
characteristics for which BDP is appropriate, it is selected. 


The results of the Business Case, capability evaluation, and front-end risk assessment, along 
with the recommended procurement strategy are referred to TBS for consideration. 
Assuming the Business Case is approved and the client department has demonstrated its 
capability to undertake the project, the client department and PWGSC are given delegated 
authority to proceed with the project within the limitations established. 


The client department and PWGSC develop and circulate a BDP-Request for Proposal (RFP) 
based on the results of the Business Case and front-end risk assessment of the investment 
project. The RFP is circulated to vendors to solicit responses. 


The client department evaluates each vendor’s proposal in accordance with the criteria and 
process specified in the RFP. A vendor is selected to proceed to the next phase. Note that as 
part of the vendor selection process, the department’s Business Case is re-validated to reflect 
vendor input, critique, and analysis. 


Contract negotiations begin. Part of the negotiations deal with how risk is allocated between 
the government and vendor. Other arrangements are also clarified, i.e., the funding scheme 
including elements of price, cost, pay back, dispute avoidance mechanism, performance 
measures, risk management plans, etc. A contract is established and the work is authorized to 
proceed to the first gate. 


At each off-ramp (which will have been previously defined in the contract), an iterative 
process is followed to ensure the Business Case remains valid, the risk profile has not 
deteriorated, and performance expectations are met. A decision is made as to whether to 
proceed or take the off-ramp. 


If the off-ramp is taken, the vendor and government sever the relationship in accordance with 
a dispute avoidance plan set out in the contract. 


10. Otherwise, this process continues until contractual expectations have been met. 
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APPENDIX B 


REVIEW OF CURRENT US DIRECTION 
IN PROCUREMENT 


BACKGROUND 


The U.S. federal government is undergoing a radical restructuring of its procurement processes. 
Large federal deficits and shrinking department budgets have forced many federal agencies to 
refocus their procurement processes in ways which allow them to maximize the value received 
from each procurement dollar. What has resulted is a new procurement philosophy that alters the 
way in which government agencies evaluate investment options, choose vendors, and execute 
contracts. This procurement philosophy is referred to as “best value” or “performance based” 
procurement. 


Essentially, best value procurement specifies that each contractor’s proposal is to be evaluated in 
terms of the expected value to be received by the government upon implementation and 
operation of the terms and conditions of the proposal. Prior to vendor selection, agencies must 
quantify the value or worth of each contractor’s proposal via a risk assessment of individual 
proposal areas in order to determine the overall merits of each proposal. After evaluating and 
comparing each proposal, the government selects that which has the strongest likelihood of 
delivering the best value to the government. 


This philosophy has received new emphasis in recent years, as embodied in the Federal 
Acquisition Streamlining Act (FASA) of 1994 and Executive Order 12931. This order 
implements the recommendations of the National Performance Review (NPR) which suggested 
that the government recognize quality factors in addition to price as well as develop “best 
practices” guidance to facilitate the use of best value procedures. Other relevant regulations and 
statutes that mandate the use of performance-based procurement are: 


e The Clinger-Cohen Act of 1996: The Clinger-Cohen Act (also known as Information 
Technology Management Reform Act) focuses on capital planning and performance-based 
management of major IT investments. 


e Government Performance Results Act (GPRA) of 1993: This act provides for the 


establishment, testing, and evaluation of strategic planning and performance measurements in 
the Federal government. 
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Performance-Based Procurement Guidelines 


In response to the new requirements outlined in the regulations and statutes described above, the 
Office of Federal Procurement Policy (OFPP) at the OMB has issued a guide to best practices for 
performance-based service contracting. This guide was designed to help federal agencies 
develop policies and procedures for implementing performance-based procurement processes for 
service contracts that satisfy regulatory and statute guidelines. Note that these guidelines are also 
quite applicable to capital investment contracting as well. In fact, they have been used and 
modified by many U.S. federal agencies to give direction to managers in evaluating and selecting 
capital related investment options. Essentially, this guide defines two elements that are necessary 
for a procurement process to be performance-based. These elements are: 


e A Performance Work Statement 
e A Quality Assurance Plan 
Performance Work Statement (PWS) 


Prior to developing a PWS, the OFPP recommends that the department or agency conduct a “Job 
Analysis”. Job Analysis involves identifying the agencies needs, the kinds of options that are 
available, and the types of outputs or results expected to be provided by the contractor. This is 
important because the outputs or results identified in the Job Analysis will form the basis for the 
performance requirements written into the PWS and verified through the Quality Assurance Plan. 


The key elements of a PWS are: a statement of the required services in terms of output, a 
measurable performance standard for the output, and an Acceptable Quality Level or allowable 
error rate. These will all have been developed or identified during the Job Analysis phase. The 
PWS describes the specific contract requirements the vendor must satisfy in performance of the 
contract. It also defines the standard of performance expected from each required task. 


Quality Assurance Plan (QAP) 


A Quality Assurance Plan, which corresponds to the stated performance standards and measures, 
is needed to determine if contractor services meet the PWS requirements. The QAP also ensures 
that a review protocol is set up to verify the performance criteria are met. Therefore, any criteria 
listed in the PWS will be reviewed using the QAP after the contract is awarded during the life 
cycle of the project. 


A QAP should clearly indicate both the surveillance schedule and method to be used. It should 
focus on the quality, quantity, and timeliness of the performance results to be delivered as 
opposed to the method in which they are to be delivered. 


These OFPP guidelines have been used also as a guide for both service contracting as well as 
capital investment contracting. Though the guidelines were originally intended only for service- 
related projects, they have been modified and used by many federal agencies for use in capital- 
related investments as well. Most of the principles relating to best-value or performance-based 
contracting implied in these guidelines are applicable to procurement processes, in general. 
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In the OFPP guidelines, the procurement process centers around a complete analysis of the 
investment options. This analysis involves examination of expected costs and benefits as well as 
relevant risk factors inherent in the implementation and operation of the investment project. 
Agencies quantify the expected value or worth relating to each proposal and choose that which 
has the greatest likelihood of delivering best value to the government 
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APPENDIX C 
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BUSINESS CASE METHODOLOGY, ANALYSIS AND 
RESULTS 


This Appendix is presented in four sections and three Annexes. Section Cl presents the 
methodological framework within which the BTP business case is examined, including the 
framework for risk analysis. Cost and schedule analysis and projections are given in Section C2 
while Section C3 follows with the analysis and outlook for benefits. Risk analysis and business 
case outcomes are presented in Section C4. 


The three annexes provide detailed technical documentation. Annex C-I displays the full 
specification of the Business Case Assessment Model. Annex C-II gives the membership of the 
two risk analysis panel sessions conducted in January 1999. A comprehensive listing of all 
assumptions, forecasts and risk assessments used in executing the business case analysis is given 
in Annex C-Il. 
C1 METHODOLOGICAL FRAMEWORK 
The methodological framework consists of six elements, as follows: 

1. Business case evaluation criteria; 

2. Specification of the base case; 

3. BTP definition; 

4. Business model development; 

5. Model Quantification and Risk Analysis; and 

6. Business Case Assessment 
Business Case Evaluation Criteria 
Two principal assessment criteria are used to gauge the business case for Net Present Value 
(NPV), one a measure of worth (value for money), the second a measure of timing. The key 
measure of worth is “net present value,” defined as the present-day value of total life-cycle costs 
minus the present-day value of total life-cycle benefits. A positive net present value means that 
the project earns more than the opportunity cost of capital and thus yields value for money to the 


province and MCSS clients. A negative NPV implies the opposite. The timing criterion asks 
whether an estimated positive NPV (positive net benefits) is likely to be achieved earlier or later 
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in the project’s life-cycle. The expectation of early net benefits implies less risk of business 
failure than would the expectation of later net benefits. 


Base Case Definition 


The base case is defined as the hardware, software and business processes that would most likely 
to be in place if BTP were not to be implemented. The base case reflects on-going initiatives to 
enhance MCSS productivity and effectiveness in the administration of social assistance, 
including the ASAP investment program. 


BTP Definition 


The analysis framework defines the BTP project in relation to the goods and services to be 
acquired in its development, implementation and operation. In contrast, the “Service Delivery 
Model (SDM)” defines BTP according to engineering system design criteria, a definitional 
framework that reflects “functionality” rather than goods and services. The business perspective 
however requires the assignment of costs and benefits to the specific hardware, software, design, 
training, administrative and maintenance goods and services to be purchased in developing and 
executing the system design. 


The business perspective must of course recognize the underlying design framework within 
which BTP goods and services are employed; it is the design framework that will determine the 
nature and magnitude of BTP benefits. The project definition, presented in Table 1, provides a 
cross-reference between ten functional design elements (that taken together represent the total 
BTP system), and the goods and services required to execute each design element. 


Model Development 


The business case assessment model is designed to represent and quantify all cause-and-effect 
relationships that link BTP goods and services to their effects; benefits; costs; risks; and business 
case outcomes. 


The business case model, displayed in Figure 1, specifies the cause-and-effect linkages between 
BTP goods and services and their effects on MCSS productivity, program spending, service 
quality and social impacts These effects are subsequently translated into monetary equivalents. 
The model then specifies BTP life-cycle costs following which costs and benefits are compared 
in relation to the business case criteria. Risk analysis is applied throughout in order to anticipate 
the effects of uncertain technical, economic and financial assumptions and forecasts on the 
business case outcomes. 
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Table 1: PROJECT DEFINITION FOR BUSINESS CASE ANALYSIS 

























Uses ACD (Automatic Call 
Distribution) and Call Routing 
technology, Telephony 
Equipment with software 
configuration work 

e Consulting hours 

MCSS hours 

Interactive Voice Response 

e Consulting hours 

MCSS hours 

Interactive Voice Response 

e Consulting hours 

MCSS hours 

Mainframe, Database, 
Custom Interface software 

e Consulting hours 

e MCSS hours 


e Change in screening process 
from interview to a two step 
(telephone plus visit) intake 










1) Centralized Intake Screening 
Unit 






















Automated Client Inquiry 


2) Automated Client Access to R 
esponse 


Information System 













Automated Client Earnings 


3) Automated Income Reporting Recoring 


System (via IVR) 















Access to Verification 
Information outside of MCSS 
e.g., Revenue Canada, EI, 
CPP, Equifax 

e Requires Agreements with 

Information Holders 

Redesign of OWT interfaces 

to provide improved and 

simplified data capture and 
usability 


















4) External Database Interfaces 




























Replace OWT software, 
Data Capture, Look up of Data 
(e.g. supports, jobs) 

e Consulting hours 

MCSS hours 

Primarily software 
application development 

e Consulting hours 

MCSS hours 






5 a) Case Management — 
Employment Assistance 









Automation-Letter, Forms, 
Bring Forwards, Auto- 
Holds/Terminate 

e Process: Eligibility Reviews, 

Client Changes, Intake 

Verification 

Centralized data repository 

and the application to 

determine eligibility and 
calculate entitlements 
including overpayment and 
arrears calculation, and tax 
table. 

Automated client priority 
assessment and eligibility 
review 

e Organizational change in 
staff focus (e.g. prioritize work 

vs. sequential work) 










5 b) Case Management — 
Financial Assistance 














Centralized (actual or 
virtual) database, custom 
eligibility and entitlement 
engine 

e Consulting hours 

MCSS hours 


















6) Centralized File Management 
and Eligibility and Entitlement 
Calculation Software 


















Automation technology 

e Process change and software 
application design 

e Consulting hours 

MCSS hours 













7) Priority-Based Assessment 
System 
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Table 1: PROJECT DEFINITION FOR BUSINESS CASE ANALYSIS (con’t) 


8) Performance Management 
System 


9) Payment and Financial 
System 


10) Disability Adjudication Unit 


Notes to table: 


Development of 
management reports, 
standards, triggers/thresholds, 
and actions 

Summarizes Key 
Performance Indicators 
including service performance 
effects 

Change in supervisory focus 

Payment production 

DBD Financial Linkages 

Overpayment Collection 

Financial housekeeping 
(General Ledger etc.) 


Process to make more 
consistent the application of 
disability eligibility 


Database Access and report 
creating technology 

Requires shift to 
performance oriented culture 
(e.g. the use of empirical data 
to manage issues/risks) 

Consulting hours 

MCSS hours 


Developed from off the 
shelf technology i.e. 
SAP/Peoplesoft (Basecase 
CIMS) 

Mass reporting — Payments 

Mainframe processing 

Consulting hours 

MCSS hours 


Organizational Change 
Process Change 
Tracking Software 
Consulting hours 
MCSS hours 





1) Change Reporting 2 consists of auto-holds. This software, a subset of 7) Case Management Software, has been 
introduced as a stand-alone component and will be incorporated into the SDM. 
2) Tax Tables consists of an automated calculation which adjusts clients tax status. This software, a subset of 7) 
Case Management Software, has been introduced as a stand-alone component and will be incorporated into the 


SDM. 


3) Consolidated Verification Process (CVP) is the process change component of the Priority-Based Assessment 
System, and will be broadened within the SDM (i.e. assessment will be conducted during Intake Verification and 


the process will be automated). 
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FIGURE 1: BTP BUSINESS CASE MODEL DESIGN 


(or ee 


Level 1 BTP 


: Architecture 
BTP Goods and Services Technology 
(Hardware, Software, Business Data 
Process Redesign) 










































































Level 2 
Implementation 
Services 
Level 3 Administrative Program Service Quality Social Effects 
——————— Productivity Payment Performance 
Performance Performance Effects 
Effects/ Effects Effects 
Effectiveness metric a metric a 
metric a metric a metric b metric b 
metric b metric b metric c metric c 
metric c metric ¢ etc. etc. 
etc. etc. 
Level 4 

















Dollar-Valuation of 
Social Effects 


Dollar-Valuation of 
Payment Savings 





Value of | BTP | 
Effects 


Dollar-Valuation of 
£ Productivity Effects 
(Benefits) 


Level 5 


Capital Costs 











Level 6 
- Sunk 
O & M Costs Life-Cycle Costs -Operation 
-Maintenance 
Level 7 
Business Case 
Outcome 
Level 8 
: Multi-Party ROI; CPP Financial Arrangement: Outcomes 
Allocation of oa 9 vtu 
RO| 
Level 9 
‘ Probability Assessment for Each Business Case Outcome 
Risk Analysis 
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BTP Effects and Benefits. As shown in Level 3 of the model in Figure 1, goods and services 
are linked to BTP effects in four categories: 


e Administrative productivity; 
e Program expenditures; 

e Client service quality; and 

e Social impacts. 


BTP is designed to improve administrative productivity in various ways. Automation is expected 
to permit workers to conduct more case-related tasks (intake, eligibility monitoring and so on) 
per hour and to require fewer overhead support services. Program expenditures are expected to 
decline as a result of more timely and accurate casework, including more accurate detection of 
non-legitimate eligibility applications and benefit claims and earlier detection of cases for 
termination. 


BTP is designed to improve client services in relation to the complexity, timeliness and 
accessibility of the application process, the accuracy and on-time receipt of payments, and so on. 
Social impacts are expected to follow accordingly in the form of more ready access to health and 
nutritional needs and the ability to spend more time in the search for productive work in lieu of 
time spent in the social assistance bureaucracy. 


The model framework identifies operational measurement metrics in each of the four categories. 
The metrics are “operational” in the sense that they lend themselves to quantification and 
subsequent valuation in Level 4 of the model. The operational metrics are presented in Figures 
1A, 1B, 1C and 1D. 


Costs and Business Outcomes. Levels 5 and 6 of the model represent BTP capital and 
operating costs, respectively, while Levels 7, 8 and 9 represent the overall business case 
outcomes including business case outcomes for each of the two parties and the business case risk 
analysis. 


Roll-out, Ramp-up and Steady-State Effects. The model framework distinguishes between 
steady-state effects and costs on the one hand, and the time needed to achieve such effects and 
realize the costs on the other. As shown in Figures 2A and 2B (for benefits and costs, 
respectively), the model distinguishes between, (i) the time needed to complete the development 
and roll-out process and thus “start-up” a new BTP project element, (ii) the time needed to 
“ramp-up” the element up to a fully 
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FIGURE 1A - ADMINISTRATIVE PRODUCTIVITY PERFORMANCE EFFECTS 


: inistrative Pr ivi in 
Total Number of FTEs for Intake Screening 


f ini ive P ivi i im 
Actes Gidh Elles Total Number of FTEs to Access Case Files 


¢ ive P ivi in 
Total Number of FTEs for Verification 


ffect 4: inistrative Pr ivi in Autom 
Eorms: Total Number of FTEs to produce Letters and Forms 














ff istrative Pr ivi in A 


ovstoava ent Cal culations; Total Number of FTEs to Buti 


Overpayment Calculations 


ff : inistrative Pr ivi in ion in Tim 
Answering Questions: Tota! Number of FTEs to Answer Questions 


fect 7; Administrative Pr ivi in 
{VR: Tota! Number of FTEs for Income Reporting 


f ini i ivi i limination of 
Financial Assessm nant Number of FTEs for Disability Adjudication Unit 


Number of FTEs for OW Employment Supports 


Effect 10:Administrative Productivity Gain - Performance Management 
System: Number of FTEs 


T Administrativ ivi im 
Administrative FTEs for the Subject Tasks 





Difference Between Base and BTP is Savings 
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FIGURE 1B - PROGRAM PAYMENT PERFORMANCE EFFECTS 


: Proar 


. : 
nt for Cli ransferring Programs Number of Overpayment 
Cases Captured per year 


ff : Program Savin R ion in th n 
Rate of ineligible Cases; Number of Ineligible Cases Accepted per year 


Effect 13: Program Savings due to Reduction in Entitlement 
Overpayment; Number of Cases with Incorrect Entitlement Detected 


Effect 14: Program Savings due to Disability Determination: Number of 
ODSP Expenditures 


ff 15: Program Savings - Performance Managemen 
Number of Overall Payments 


Total Program Savings due to a Reduction in Total Number of 
Overpayments and Ineligible Cases Receiving Entitlements per year 





Difference Between Base and BTP is Savings 
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FIGURE 1C - SERVICE QUALITY PERFORMANCE EFFECTS 





ion in Ineligible Cli ivin itlem Number 
of Correct Eligibility Decisions made as a Percentage of Total Eligibility 
Decisions (%) 


f 17: ion of Incorrect R rali i 
Management: Number of Cheques Distributed to the Correct Address on 
Record as a Percentage of Total Cheques Distributed (%) 


Effect 18: Reduction in the Distribution of Incorrect Entitlements Number 
of Payment Cheques Written in the Correct Amount as a Percentage of 
Total Cheques Written (%) 


Effect 19: Reduction in Late Cheques Number of on Time Cheques as a 
Percentage of Total Cheques Distributed (%) 





Difference Between Base and BTP is Savings 
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FIGURE 1D: SOCIAL IMPACT EFFECTS 


Telephone Screening 


Custom Calls Travel time saved in both|Saved travel costs and _ |New referral service on 
directions for applicants {baby-sitting costs for telephone will save time 
and additional non- applicants with by directing applicant to 
professional helpers. dependent children, and |the right service, 

Also, time saved by not {without sitter. including those outside 
waiting in office and BTP. 


Intake Verification 


Phone time i With faster financial 
waiting at home. Also, verification, there will be 
re-applicant's time saved more time for 
by not re-entering their employment counseling 
data. that improves job- 

hunting success. 


3rd party information |Client's time saved by Saved travel costs and _ {Societal benefit from 
verification having on-line external baby-sitting costs for more time devoted to 


links. They don't have to |applicants with take care of children 
visit other offices to get {dependent children and 
required records. without sitter 

Case Management 

Client Enquiries Time saved for the Saved travel costs and _|Belief that the new 
proportion of clients that |baby-sitting costs for system is more subjective 
no longer need to go to_japplicants with and reliable brings social 
office to check status. dependent children and benefits of less worry 

without sitter 

Changes in Time saved for office Saved travel costs and |Confidence gained from 

Circumstances visit for those who have |baby-sitting costs for client's ability to serve 
changed circumstances applicants with themselves 
that requires the updating |dependent children and 
of records. without sitter 

Eligibility Payment 


processing 
Repaying overpayment 
and interest may be 


hardship for some 
clients. 
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operational state through training, the shake-out of bugs and so on, and (iii) the steady-state level 
at which the element will perform beyond the ramp-up period. 


Quantification and Risk Analysis 


The detailed cause-and-effect relationships that link BTP goods and services to their effects and 
costs are specified in detailed structure and logic models and subsequently coded for computer 
simulation. Figure 2 presents an example; the full array of structure and logic models is given in 
Annex C-I. 


Each variable in the structure and logic models is quantified for each year in the project life-cycle 
(1996 to 2003, the fullest life of the BTP CPP agreement). In order to account for uncertainty 
and facilitate risk analysis, each variable is given three annual values, the median expected 
outcome and a range defining the 80 percent probability interval associated with median. Figure 
4 gives an example. The risk analysis framework and computer program called RAP (Risk 
Analysis Process) is used to translate the three values (known as “risk markers’’) into associated 
probability distributions, as shown in Figure 4A. 


Median values for each variable in the model are drawn from data developed by the BTP team in 
the course of various task orders, with particular reference to the work conducted under Task 
Order 4 and subsequent MCSS updates. The 80 percent probability ranges are developed in two 
steps. Step 1 is a risk assessment for each variable conducted by HLB staff engineers and 
economists. 


With a view to obtaining stakeholder input and consensus on the models and risk assessments, 
Step 2 involves facilitated panel scrutiny of both the HLB structure and logic models and the 
HLB risk assessments. Two panel sessions (RAP Sessions’) were conducted. The first session 
was devoted to the structure and logic models (January 18, 1999). The second (January 28, 
1999) focused on the risk assessments. Panelists were drawn from senior BTP management 
representing both MCSS and Andersen Consulting. Table 2 presents the factors used by HLB 
and panelists as the basis for assessing and quantifying uncertainty ranges for each variable. 
Annex C-II lists the participants in each of the two panel sessions. 


Business Case Assessment 


Once the structure and logic models are specified, quantified and scrutinized, business case 
results are obtained. Computer simulation of the model framework is used to generate ten 
thousand possible outcomes based on random simulations from the risk assessments for each 
variable in the model (the process, called Monte Carlo simulation, is illustrated in Figure 5). The 
frequency with which outcomes occur and 
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TABLE 2: OVERVIEW RISK ASSESSMENT FACTORS 


DESIGN/ROLL-OUT 
SHAKE-OUT 

HUMAN FACTORS 
LABOUR RELATIONS 
OBSOLESCENCE 
FISCAL PRIORITIES 
FUNCTIONALITY 
SOCIAL POLICY 
PARTNERING 

SKILL MIX AND TIMING 
STAFF TURNOVER 
MEASUREMENT RISK 





DESIGN/ROLL-OUT 
e Lower 10 Percent. Probability that strengths in design/roll-out or prototyping will 
accelerate development and thus accelerate start-up; 
e Upper 10 Percent. Probability that weaknesses in design and/or roll-out or prototyping 
of architecture/ technology/business process will delay initiative development and thus 
delay start-up. 


SHAKE-OUT 
e Lower 10 Percent. Probability that performance strengths will shorten ramp-up; 
e Upper 10 Percent. Probability that performance deficiencies with newly in-place 
technologies or business processes will lengthen ramp-up. 


HUMAN FACTORS 
e Lower 10 Percent. Probability that strengths in learning, acceptance and take-up 
strategy, or higher than expected worker reception levels will shorten ramp-up; 
e Upper 10 Percent. Probability that weaknesses in learning, acceptance and take-up 
strategies and lower than expected worker reception levels will lengthen ramp-up. 


LABOUR RELATIONS 


e Lower 10 Percent. Probability that strong labour relations will shorten start-up and 
reduce ramp-up; 


e Upper 10 Percent. Probability that provincial labour unrest will delay start-up or 
lengthen ramp-up. 
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TABLE 2: OVERVIEW RISK ASSESSMENT FACTORS (con’t) 


OBSOLESCENCE 
e Lower 10 Percent. Probability that stable technology will increase steady-state 
performance; 


e Upper 10 Percent. Probability that rapid technological change will lengthen initiative 
development, delay start-up or lengthen ramp-up. 


FISCAL PRIORITIES 
e Lower 10 Percent. Probability that budgetary shifts will accelerate development, 
accelerate start-up or shorten ramp-up; 
e¢ Upper 10 Percent. Probability that shifts in fiscal and budgetary priorities will delay 
initiative development and delay start-up or lengthen ramp-up. 


FUNCTIONALITY 
e Lower 10 Percent. Probability that weaknesses in functionality will diminish steady- 
state. Probability that innovation and strength in functionality will shorten ramp-up; 
e Upper 10 Percent. Probability that weaknesses in functionality will increase steady- 
state. Probability that innovation and strength in functionality will lengthen ramp-up; 


SOCIAL POLICY 
e Lower 10 Percent. Probability that stable social policy will accelerate start-up, or 
shorten ramp-up. Probability that changes in eligibility or entitlement policy will shift 
administrative requirements beyond BTP functionality and thus diminish steady state; 
e Upper 10 Percent. Probability that stable social policy will increase steady state. 
Probability that changes in eligibility or entitlement policy will shift administrative 
requirements beyond BTP functionality and thus delay start-up, or lengthen ramp-up. 


PARTNERING 

e Lower 10 Percent. Probability that weaknesses in vendor/ government risk-sharing 
relationship will lengthen initiative development and thus diminish steady-state. 
Probability that strengths in vendor/government risk sharing relationship will accelerate 
development and thus accelerate start-up, shorten ramp-up; 

e Upper 10 Percent. Probability that weaknesses in vendor/government risk-sharing 
relationship will lengthen initiative development and thus delay start-up, or lengthen 
ramp-up. Probability that strengths in vendor/government risk sharing relationship will 
accelerate development and thus increase steady-state. 
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FIGURE 2A: TIMING FRAMEWORK FOR EFFECTS AND BENEFITS 


ARCHITECTURE SPECIFICATION COTS OR CUSTOM 
Level 1 
Hardware Hardware Mode COTS (Custom off the 
Shelf) 
Software Software Mode COTS with some 
Customization 
Business Process BPR Mode 
Level 2 Training/Integration Training Mode All Custom 


START-UP 
Median: Expected Month/Year of Initiation 
Risk Markers: 10 percent lower bound; 10 percent upper bound 


RAMP-UP 
Median: Expected Months to Steady-State 
Risk Markers: 10 percent lower bound; 10 percent upper bound 


STEADY-STATE EFFECTS 
Level 3 Median: Percent Increase or Decrease in Metric 
Risk Markers: 10 percent lower bound; 10 percent upper bound 


BTP BENEFIT 
Level 4 Median: Monetary Value of Effect 
Risk Markers: 10 percent lower bound; 10 percent upper bound 





SS 
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ARCHITECTURE 


Level 1 
Hardware 


Software 


Business Process 


Level 2 


Level 5 and 6 


SUNK 
GOING FWD 


Level 7 and 8 


Level 9 
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Training/Integration 


CAPITAL COSTS 


Year 1, Year 2, Yeart 
Year 1, Year n+1, Yeart 


FIGURE 2B: TIMING FRAMEWORK FOR COSTS 


SPECIFICATION 


COTS OR CUSTOM 


Hardware Mode 
Shelf) 

COTS with some 
Customization 


Software Mode 
BPR Mode 


Training Mode All Custom 


DISCOUNT/INTEREST RATE 
Median 
Risk Markers: 10 percent lower bound; 10 percent upper bound 


PROJECT LIFE 
Median 
Risk Markers: 10 percent lower bound; 10 percent upper bound 





COSTS 
IMPLEMENTATION/ 
TRAINING 
Year 1, Year 2, Yeart 
Year 1, Year n+1, Yeart 


Year 1, Year 2, Yeart 
Year 1, Year n+1, Yeart 


Median: Percent Increase or Decrease in Metric 
Risk Markers: 10 percent lower bound; 10 percent upper bound 









BTP BUSINESS CASE OUTCOMES 
Return on Investment 
First-Year ROI 
Net Present Value 
Pay-Back 


Median, Mean, Probability Range 


Muli-Party ROI; CPP Financial Arrangement: Outcomes 





COTS (Custom off the 


CAPITAL REFRESH O&M 





Year t through n 
Year t through n 
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FIGURE 3: EXAMPLE OF SUB-MODEL STRUCTURE AND LOGIC 


Functional/Process Group: Intake 


BTP Integrated Sub-Initiative: 


Effect 1: 
Metric: 


Total Number of Administrative FTEs 









BTP 


Screening Time 
per OW 
Application 


BTP 


Number of OW 
Application Calls 
per year 





Number of OW 
Application Calls 
per year 






Product Product 





Administrative Productivity Gain Due to Improved Screening 








Centralized Intake Screening Unit 


Screening Time 


per OW 
Application 


Total Time to Productive Total Time to Productive 
Screen OW Minutes per FTE Screen ODSP Minutes per FTE 
(minutes/year) (minutes/year) 
Divide by Divide by 











Total Number of 
OW FTEs for 
Intake Screening 







Clerical and IMO Product 


Gross Up, OW 


and ODSP 








Total Number of 
FTEs for Intake 
Screening 


Net: Effect due to BTP 






Level 3A 


Reference: 1.1.1 Social Assistance Blueprint - Implementation Plan and Business Case 


Hickling Lewis Brod Inc 


Total Number of 
ODSP FTEs for 
Intake Screening 
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Figure 4A: QUANTIFYING VARIABLE - AN EXAMPLE 


Capital Expenditure 
($millions) 


/ mepian | 10% LOWER* 
[oss [oso | tz 


* INDICATES THE UPPER AND LOWER LIMITS OF AN 80% 
CONFIDENCE INTERVAL. 
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Figure 4A: RISK ANALYSIS - AN EXAMPLE 


Capital Expenditure 
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Figure 5: MONTE CARLO SIMULATION 


A TECHNIQUE FOR COMBINING PROBABILITIES 


Months to ae 

90% Productivity ds aay me 
(months) 

Risk of Capital 
Implementation Expenditure 
Delay (months) ($) 
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Costs of Technological 
Implementation 
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Figure 6: RISK ANALYSIS RESULTS CHART 


NPV of Technological Implementation 


Probability of Exceeding 


100 
Net Present Value ($millions) 
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recur in the sampling process forms a probability distribution, or risk analysis, of BTP’s effects, 
benefits, costs and business case outcomes (as illustrated in Figure 6). 


BTP COST AND SCHEDULE ANALYSIS 


BTP project costs consist of one-time hardware and software expenditures, on-going hardware 
and software leasing expenses, development and testing facility costs, operating and maintenance 
expenses, and hourly costs for professional services (design, project management, training and 
other professional service costs). Table 3 presents the median cost estimates for the year in 
which they are expected to occur. The median cost estimates presented in Table 3 coincide with 
the planned schedule for design, build, roll-out and ramp-up activities presented in Figure 7. 


Risk assessments for each variable in the cost and schedule estimating models are presented 
Annex C-II. 


BTP EFFECTS AND BENEFITS 


The operational measurement metrics (Figures 1A through 1D) in each of the four BTP impact 
categories (productivity, program expenditures, service quality and social impacts) are translated 
into monetary equivalents on the basis of assumptions given in Annex C-III. These assumptions 
were developed by HLB and scrutinized in the second of the two stakeholder panel sessions 
(January 28, 1999). 


Monetary Valuation of Productivity Gains 


Productivity gains are valued as the difference between MCSS FTE requirements under the base 
case versus the BTP case. MCSS wage rates, benefits and overheads are used to value FTE 
requirements, by category. Although reductions in FTE requirements may not be translated into 
provincial staff reductions, it is assumed that reduced FTE requirements would be translated into 
a more efficient allocation of staff resources within MCSS and among provincial government 
ministries. The alternative assumption, that excess labour requirements are not re-optimized but 
carried in the form of waste, cannot be justified either theoretically or by reference to the 
observed behaviour of governments in the face of FTE efficiency gains. 


Monetary Valuation of Program Savings 


Program savings are estimated on the basis of projected caseload and entitlements per case in the 
base case versus the BTP case. Through more effective screening, monitoring and verification, 
BTP is expected to reduce both caseload and entitlements per case. Annex C-III presents the 
median projections and risk assessments for each variable in the caseload and entitlement 
forecasting models. 
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Figure 6: RISK ANALYSIS RESULTS CHART 
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recur in the sampling process forms a probability distribution, or risk analysis, of BTP’s effects, 
benefits, costs and business case outcomes (as illustrated in Figure 6). 


BTP COST AND SCHEDULE ANALYSIS 


BTP project costs consist of one-time hardware and software expenditures, on-going hardware 
and software leasing expenses, development and testing facility costs, operating and maintenance 
expenses, and hourly costs for professional services (design, project management, training and 
other professional service costs). Table 3 presents the median cost estimates for the year in 
which they are expected to occur. The median cost estimates presented in Table 3 coincide with 
the planned schedule for design, build, roll-out and ramp-up activities presented in Figure 7. 


Risk assessments for each variable in the cost and schedule estimating models are presented 
Annex C-H. 


BTP EFFECTS AND BENEFITS 


The operational measurement metrics (Figures 1A through 1D) in each of the four BTP impact 
categories (productivity, program expenditures, service quality and social impacts) are translated 
into monetary equivalents on the basis of assumptions given in Annex C-III. These assumptions 
were developed by HLB and scrutinized in the second of the two stakeholder panel sessions 
(January 28, 1999). 


Monetary Valuation of Productivity Gains 


Productivity gains are valued as the difference between MCSS FTE requirements under the base 
case versus the BTP case. MCSS wage rates, benefits and overheads are used to value FTE 
requirements, by category. Although reductions in FTE requirements may not be translated into 
provincial staff reductions, it is assumed that reduced FTE requirements would be translated into 
a more efficient allocation of staff resources within MCSS and among provincial government 
ministries. The alternative assumption, that excess labour requirements are not re-optimized but 
carried in the form of waste, cannot be justified either theoretically or by reference to the 
observed behaviour of governments in the face of FTE efficiency gains. 


Monetary Valuation of Program Savings 


Program savings are estimated on the basis of projected caseload and entitlements per case in the 
base case versus the BTP case. Through more effective screening, monitoring and verification, 
BTP is expected to reduce both caseload and entitlements per case. Annex C-III presents the 
median projections and risk assessments for each variable in the caseload and entitlement 
forecasting models. 
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Table 3: BTP PROJECT BUDGET 
(in millions of current dollars) 


One-Time 
Hardware 
One-Time 
Software 
Ongoing 
Hardware 


Ongoing 
Software 
License Fees 
Operating 
and 
Maintenance 
Charges 
Development 
Environment 
Costs 





Sarre paar se ee 


Source: BTP Project Team, December 1998 
Note: Cost ecalation factors range between five and eight percent, depending upon category. 
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Whereas HLB adopted median projections of caseload effects based on estimates developed by 
the BTP project team, we conducted an independent analysis as the basis for the risk assessment 
of caseload effects. | 


HLB Statistical Audit of Caseload Terminations Attributable to BTP. As the basis for our 
independent analysis, we developed a multivariate statistical model of monthly case terminations 
between April 1996 and November 1998. We _ used this model to measure the quantitative 
magnitude and statistical significance of any increase in the number of monthly terminations 
attributable to CR1 and CR2 and to CVP “early opportunity investments” carried out under the 
BTP project. Our findings indicate that terminations due to CR1 and CR2 increased on average 
by 28 percent per month between September 1997 and November 1998. As of November 1998, 
however, no increase in terminations were statistically attributable to CVP. These findings led 
HLB to estimate a downward skew in the probability range for projected CVP benefits. 


The HLB multivariate statistical model of terminations is reflected in three equations, as follows: 
LTERMIN = BETAp + BETA,;*TREND + BETA2*STRIKE] + BETA3*STRIKE2 + 
BETA4*CR1CR2 + BETAs*CVP + BETA.6*SPP + BETA7*OSAP = 


LSSPTERMIN = BETA, + BETA,;*TREND + BETA 2*STRIKE!] + BETA3*STRIKE2 + 
BETA,*CVP + BETAs*SPP + BETAs*OSAP wae 


LSSPTERMIN = BETA), + BETA);*TREND + BETA2*STRIKE!] + BETA3*STRIKE2 + 
BETA,*CR1ICR2 + BETAs*CVP + BETA.¢*SPP + BETA7*OSAP ef 


where: 

LTERMIN denotes Log of Total Terminations 
LSSPTERMIN denotes Log of SSP Terminations 
TREND denotes Linear Time Trend 


STRIKE1 denotes a dummy variable for the Provincial Employee Strike set equal to one in 
March 1996 and zero otherwise 


STRIKE2 denotes a dummy variable for the Postal Strike and Ice Storm set equal to one in 
December and January 1998 and zero otherwise 


CR1CR2 denotes a dummy variable for the introduction of Change Reporting I and I, set equal 
to one from September 1997 onwards, and zero otherwise 
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CVP denotes a dummy variable for the introduction of CVP, set equal to one from April 1998 
onwards and zero otherwise 


SSP denotes a dummy variable for the transfer of the Sole Support Parents (SSP) program to 
OW, set equal to one in November 1998 and zero otherwise 


OSAP denotes a dummy variable for the OSAP Income Preclusion of Welfare Receipts, set 
equal to one for September and Nov 1996 and zero otherwise 


The Ordinary Least Squares results are given in Table 4. Equation 1 explains 80.2 percent of the 
monthly variation in total terminations between January 1996 and November 1998. The trend 
term indicates a downward trend in terminations of 1.6 percent per month, after allowing for 
strikes and other “shocks.” The introduction of CR1 and CR2 is measured to yield a statistically 
Significant coefficient that translates into a 28 percent monthly impact on terminations (the 
calculation is (eneae - 1) * 100 = 27.9 percent). The coefficient for CVP, on the other hand, is 
Statistically insignificant. Equations 2 and 3, which restrict the sample to the SSP caseload, still 
indicate a statistically insignificant effect of CVP on case terminations. 


The results outlined above do not necessarily imply that CVP has failed to generated positive 
results on SSP terminations. A full year or so of experience may reasonably be required in order 
to ascertain such proof. The absence of confirmed CVP benefits as of November 1998 is thus 
notable though not alarming. 
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Table 4 Ordinary Least Squares Results for Equations 1-3 (January 1996-November 1998) 


(192,041) Cito) (E2192) 
(-4.609) (-1.851) (-3.742) 
| EMIS (-3.162) (-3.999) 
(-3.016) | (-1.570) (-2.709) 
(3.182) (3.178) 
(0:373)ie (O67) sels, (-0.226) 
(4.288) (3.900) (4.786) 
(5.247) (4.233) (5.149) 


Diagnostic Statistics 
0.802 0.670 0.760 


R2 
F — Statistic 15.645 9.481 | asia OD 
Durbin-Watson Statistic 1.687 1505 1.306 












Independent Variable 




































Monetary Valuation of Social Impacts 


The projected social impacts of BTP are estimated on the basis of client time savings (in hours) 
and the economic value of these time savings (in dollars per hour). The methodology for 
estimating client time savings is given in Table 5. 


The hourly economic value of client time savings is assumed to be reflected in the provincially 
established minimum wage level of $6.85 per hour. This assumes that the minimum wage level 
reflects the provincial government’s belief regarding the average opportunity cost of time among 
individuals who bear the risk of requiring social assistance. An alternative valuation assumption 
would equate the social value of time with the average entitlement per case under the ODSP and 
OW programs. On a per hour basis (taken over an eight hour day), the combined ODSP and 
OW social assistance rate averages $6.27 per hour. However, since the differential between this 
rate and the minimum wage embodies a financial incentive for recipients to return to the 
workforce as soon as possible, it is assumed here that the more accurate reflection of social value 
is embodied in the minimum wage level. 


ee ee a ga ee an ee SS 
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Central forecasts, presented in Table 6, indicate that BTP will yield an additional $53.9 million in 
annual social benefits to recipients of OW and ODSP once BTP has reached steady-state 
performance. The present value (in 1996) of these benefits is $49.5 million. 


BUSINESS CASE OUTLOOK AND RISK ANALYSIS 
Expected business returns -- returns arising under central forecasts of BTP costs, schedule and 
performance — are evaluated in terms of (i)life-cycle net benefits; (ii) return on investment and 


timing; (iii) net benefits at “steady-state” (full BTP functionality); (iv) the effect of cost sharing 
with the private sector; and (v) risk. 
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Table 5: Valuation of BTP Social Impacts - Methodolog 

ae ee Teens) SkeTime CostSavines 10) | Out of Pocket Cost Savings Intangible Benefits 

Intake. =| radii E-@h2 ai atPomad| soar Yo (8007 of) sulev tiinevin adT: _domeerreene| 
PR eer eee a ee ee Se ee ee ee 


Saved travel costs and baby- |New referral service on 


Custom Calls Travel time saved in both 

directions for applicants and _|sitting costs for applicants with|telephone will save time by 
additional non-professional {dependent children, and directing applicant to the right 
helpers. Assumes help can be |without sitter. service, including those 
obtained on phone or from 
home. Also, time waiting in 
office and queuing is 


eliminated. 


Intake Verification | = —idzC 


le 
Phone time For those not screened out, 
and had a home visit, will no 
longer need to wait at home. 
Also, re-applicants will 
already have their data in 
databases 
Clients that don't have the 
required forms must visit, on 







































With faster financial 
verification, there will be more 
time for employment 
counseling that improves job- 
hunting success. 





















3rd party information 
verification 


Saved travel costs and baby- 
sitting costs for applicants with 





3 is used to estimate time 
saved by having on-line 
external links 


2. Cabe Management [n= «ters aw. roel aco ne) omen orcvineumenmenm ine enamine: 


Client Enquiries Estimated 10% of clients will | The saved cost of visit is the |Belief that the new system is 
not need to go to office. The |Custom call out-of-pocket cost |more subjective and reliable 
calculation of saved cost of a |proportioned by number of brings social benefits of less 
Visit is the custom call time cases. worry and aggravation. 
cost proportioned by number 
of cases. 
Those who have changed 
circumstances save the cost of 
a visit. The calculation uses 
the custom call time cost 
proportioned by number of 
cases. 
































































Changes in 
Circumstances 


The saved cost of visit 
calculation uses the custom 
call out-of-pocket cost 

proportioned by number of 
‘Icases. 







of system; proper safety net; 
ability to serve selves; more 
time to take care of children 
With an average 5% and 
maximum 10% interest rate 
there may be hardship for 
some to payback an 
overpayment. 





























Over/Underpayment 
reduction 


An estimated 6% got at least 
one underpayment in the year. 
The calculation of average 

welfare loss uses the proxy of 
the minimum wage for the 

period the outstanding amount 
is delayed. 
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Table 6: Valuation of BTP Social Impacts — Results (in dollars per year) 


SDM Functional Areas Time Cost Savings | Out-of-Pocket Cost | Incremental Social 
Savings Benefit ($) 
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3fC_Party Information Verification 14,063,906 7,734,375 21,798,281 


~ Case Management ies CEM AUTIVE PRON OS TEETER PONT ah aR 
Client Enquiries 503,475 860,417 1,363,89 


Changes in Circumstances 4,027,800 3,412,500 7,440,300 


“Eligibility Payment cy Oa aes 
Standardized Processing th sleds all henreaveslbie 
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N 
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Over/Underpayment Reduction 1,208,304 


Total Incremental Social Benefit 


Net Benefits 


As shown in Table 7, the realization of central cost, schedule and performance projections means 
that the BTP investment program would yield $297.2.million in benefits net of all costs (in 
present-day dollars of 1996 purchasing power over the 1996-2003 life-cycle of the CPP contract, 
discounted at five percent per year). Benefits arise in the form of (i) productivity gains in the 
MCSS administration of OW and ODSP, and (ii) reductions in the volume of inaccurate 
payments under these programs. 


Table 7: BTP Life-Cycle Business Case Results (Benefits minus Costs over the Period 
April 1996 to March 2003, Excluding Social Benefits) 
( Present-day (1996) value, at five p 





| Probability | Probability | Probab 
-thatNPV | thatNPV | that NPV 
will be: - willbe: | willbe: 


Net Benefits (NPV) 
Notes: 
1) Base caseload of 591,000 cases in 1996/1997, using September 1998 forecast. 


2) Labour rates include annual escalation factors of 8% for Andersen Consulting and sub-contractors, and 5% for 
MCSS and sub-contractors (applies to costs and benefits). 

3) Cost information based on Andersen Consulting supplied workdays. 

4) Schedules developed during January 28 Session. 


5) 30% of the active caseload is assumed to have an overpayment under base and BTP figures. 
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As shown in Table 8, the business benefits of BTP are manifest in improved quality of service to 
Table 8: BTP Service Quality Effects (at Steady-State) 


OW and ODSP recipients. | 
Service Quality Performance Indicator Without With Effectiveness 
BTP BTP Improvement 
Percent of Eligibility Decisions Made Correctly 94.5% 95.9% 


Percent of Payments Made in the Correct 72.8% 90.6% 24.5% 
Amount 

Percent of Social Assistance Payments Made on 95.5% 98.4% 3.0% 
Time 


Return on Investment 

Central projections indicate a cumulative seven-year return on BTP investment of 222 percent 
(Table 9). High returns of this magnitude indicate that modernizing MCSS business processes is 
sharply overdue. Had such investment been made earlier, returns would have been lower, yet 
nevertheless acceptable, while Ontario taxpayers would have not have waited a protracted period 
of time for greater government effectiveness and efficiency in the administration of social 
assistance. 














Table 9: BTP Return On Investment, in percent (excluding social benefits) 















2002-2003 


The results outlined above account for gains in productivity and reduced program outlays. 
Central forecasts indicate that BTP will also yield an additional $49.5 million in net social 
benefits to recipients of OW and ODSP over the first seven years of the program (figures from 
Table 6 translated to present-day value at five percent discount). When the economic value of 
these social benefits is combined with gains in MCSS productivity and spending reductions, total 
net benefits rise to $346.7 million ($297.2 million plus $49.5 million). 





Steady-State Benefits 


The results above are reported on a life-cycle basis so as to account for lower benefits during the 
multi-year roll-out and ramp-up of BTP systems and business processes. Central projections 
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indicate that BTP processes will be operating in “steady-state” (full functionality and 
productivity) by year eight of the program (2004). Table 10 indicates that at that time annual 
benefits (excluding social benefits) will have reached $718.8 million while the major fraction of 
BTP costs will have been incurred in previous years. Steady-state net benefits will thus reach 
$694.3 million by year eight of the program. 


It is noteworthy that the roll-out of "early opportunity" investments (CR-I, CR-II and CVP) has 
generated positive net benefits on an annual basis (for 1997/98 and 1998/99). While going 
forward, the BTP investment program can expect two years of negative returns, the early 
"banking" of positive outcomes means that cumulative rates of return remain positive in each 
year of the seven-year development and roll-out program (see Table 9). 


Table 10: BTP Annual And Steady-State (Eighth Year) Costs, Benefits And Net Benefits 
(excluding social benefits) 


Total Benefit i in 32.0 64.7 65.2 62.0 79.1 o 5 718.8 
a a all al il a 
Total Cost in year 

(cg est oh Bl a i Mle el ia 
Net Benefit in year 26.4 | (24.8) | (14.5) | 390.1 | 694.3 
“ser (ps laid ll aa lia fi aiid 


Cost Sharing with the Private Sector 


(in millions of current aollats) 







The results outlined above account for all BTP costs and all resulting BTP benefits. However, 
present CPP financial arrangements between MCSS and Andersen Consulting entitle the 
consultant to payments of no more than $180 million, an amount that may lie beneath the 
consultant’s cost of delivering total BTP functionality at its present billing rates . If the 
consultant nonetheless delivers total functionality within the $180 million payment “Cap,” net 
benefits from the perspective of the provincial taxpayer will exceed the $297.2 million value 
cited above by the difference between the Cap and Andersen Consulting’s cost. From the 
perspective of total costs, however, regardless of whether they are borne by the taxpayer or the 
vendor, the net present value remains unchanged. 
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Table 10: BTP Annual And Steady-State (Eighth Year) Costs, Benefits And Net Benefits 
(excluding social benefits) 


(in [vues of current soets 















Total Benefit in 
Total Cost in year 
Net Benefit in year 


Cost Sharing with the Private Sector 


7 a 8 
(24.8) | (14.5) | 3901 | 69423 





The results outlined above account for all BTP costs and all resulting BTP benefits. However, 
present CPP financial arrangements between MCSS and Andersen Consulting entitle the 
consultant to payments of no more than $180 million, an amount that may lie beneath the 
consultant’s cost of delivering total BTP functionality at its present billing rates . If the 
consultant nonetheless delivers total functionality within the $180 million payment “Cap,” net 
benefits from the perspective of the provincial taxpayer will exceed the $297.2 million value 
cited above by the difference between the Cap and Andersen Consulting’s cost. From the 
perspective of total costs, however, regardless of whether they are borne by the taxpayer or the 
vendor, the net present value remains unchanged. 


Risk Analysis 


The analysis reveals that a number of material risks confront BTP management in seeking to 
achieve the central forecasts reflected above and thus deliver positive business returns. Among 
these the most prominent are (i) schedule risk; (ii) the risk of failure to achieve targeted levels of 
effectiveness; (iii) the risk of inaccurate case load forecasts; and (iv) the risk of inaccurate 
assessments of the opportunity cost of capital. 
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Schedule Risk. A dominant risk is that of slippage in the roll-out and ramp-up of key processes 
and support technology. Analysis of potential slippage risk, taken together with the effects of all 
other risk factors, indicates a five percent probability that that BTP benefits will exceed BTP 
costs by no more than $42.9 million, and a one percent risk that costs will actually exceed 
benefits by more than $8 million — (Table 7 -- in present-day dollars of 1996 purchasing power 
over the 1996-2003 life-cycle of the CPP contract, discounted at five percent per year). On the 
other hand, sound and aggressive risk management of the schedule is found to offer substantial 
rewards. As compared with the baseline schedule, each month of accelerated roll-out and ramp- 
up yields five percent greater net benefits. While aggressive risk management and mitigation of 
schedule risk itself carries a cost, the business case for such activity is very high. 


Effectiveness Risk and the Crystallization Process. Benefits in the form of improved 
productivity and reduced program outlays arise when BTP business processes trigger more 
effective MCSS performance in administering intake and eligibility monitoring and benefits- 
management activities. Delay in achieving provably greater effectiveness will delay, and 
weaken, business case returns. 


As shown earlier, multivariate statistical analysis indicates that improved effectiveness can be 
confirmed in relation to the introduction CR-I and CR-II in September 1997 and that provable 
business and social benefits are accruing accordingly. On the other hand, statistical analysis does 
not yet confirm that improved effectiveness has arisen from CVP, as introduced in April 1998. 


On the other hand, failure to ascertain the emergence of such proof with eight months of CVP 
experience should be viewed as an emerging risk and managed accordingly. Since at present the 
process of crystallizing effects and benefits, while rational and well documented, does not 
include tests of statistical proof, a such tests should be incorporated into the process as part of an 
improved risk management framework. 


Client Base. The client base is a risk factor because the benefits of BTP are found to fall more 
quickly than BTP costs as the number of active cases declines (or rise more quickly with 
unexpected growth in the number of active cases). Although case load would need to decline 
substantially more than projected under even the most optimistic economic outlook in order to 
materially dilute the business case for BTP as it currently scoped, risk managers should 
recognize the link between the net benefits of BTP and the size of the case load and seek 
opportunities to manage the timing and scope of project outlays accordingly. 


The Opportunity Cost of Capital. Results reported above assume a provincial opportunity cost 
of capital of five percent (in dollars of current purchasing power). While this is consistent with 
current market conditions, governments at times apply above-market discount rates in appraising 
public investments as a means of guarding against the risk of crowding-out more productive 
private sector investments. At a ten percent discount rate for example, the expected net present 
value of BTP reported in Table 7 falls from $297.2 million to $216.7 million. 
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Economic reasoning supports the lower discount rate in the case at-hand, however, for two 
reasons. First, the willing participation of a private sector firm in sharing the investment risk of 
BTP indicates that crowding out is not occurring. Second, economic theory supports a lower 
discount rate for social investments on the grounds that capital markets understate the rate at 
which society is willing to defer consumption in order to sustain a minimum state of health and 
nutrition for those in need. 


Conclusion 


HLB Inc. believes that the BTP investment program, in its present state of design and scope, 
offers a sufficiently high probability of positive business case returns and sufficiently low risk of 
negative returns to justify the magnitude of investment funds budgeted for the acquisition of 
associated goods and services. 


HLB Inc. analysis indicates the need for aggressive steps to manage risk and mitigate against the 
effects of uncertainty, particularly in relation to, (i) adherence to the scheduled roll-out and ramp- 
up of BTP business processes and supporting technologies, and (11) timely statistical proof of 
system effectiveness and crystallized benefit. Failure to manage and mitigate these risk poses a 
material risk that BTP will yield negative business case returns. 
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ANNEX C-I: STRUCTURE AND LOGIC DIAGRAMS OF 
BUSINESS CASE MODEL 
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Functional/Process Group: Intake 
Centralized Intake Screening Unit 


BTP Integrated Sub-Initiative: 


Effect 1: Administrative Productivity Gain Due to Improved Screening 


Metric: Total Number of Administrative FTEs 


Number of OW 
Application Calls 
per year 


Screening Time 
per OW 
Application 





Product 









Total Time to 
Screen OW 
(minutes/year) 


Productive 


Minutes per FTE 





Divide by 








Total Number of 
OW FTEs for 
Intake Screening 


Clerical and IMO Product 
Gross Up, OW 


and ODSP 





Net: Effect due to BTP 


Total Number of 
FTEs for Intake 
Screening 


Level 3A 


Number of OW 
Application Calls 
per year 








Screening Time 
per OW 
Application 





Product 










Total Time to 
Screen ODSP 
(minutes/year) 


Productive 


Minutes per FTE 





Divide by 








Total Number of 
ODSP FTEs for 
Intake Screening 


Reference: 1.1.1 Social Assistance Blueprint - Implementation Plan and Business Case 


Functional/Process Group: Case Management 
BTP Integrated Sub-Initiative: Centralized File Management and Eligibility 
and Entitlement Calculation Software 


Effect 2: Administrative Productivity Gain Due to Reduced Time to Access Case Files 
Metric: Total Number of Administrative FTEs 


Number of OW 
File Accesses per 
case 


: Number of ODSP 
Time to Access s 
File, OW File Accesses per 
case 


Time to Access 
File, ODSP 





Product Product 






Bb 





Total Time to Total Number of Time to Total Number of 
Access File. OW Current OW Access File, Current ODSP 
: Cases ODSP Cases 


(minutes/case) 





(minutes/case) 





Product Product 









Total Time to 
Access Files per 
year, OW 


Productive 
Minutes per FTE 


Total Time to Productive 


year, ODSP 





Divide by Divide by 


IMO Gross Up, Product 


OW and ODSP 





: Total Numb f 
Net: Effect due to BTP Teenie keke 


Files 





Level 3A 


Reference: 1.2.4 Social Assistance Blueprint - Implementation Plan and Business Case 


Access Files per Minutes per FTE 


Functional/Process Group: 
BTP Integrated Sub-Initiative: 


Intake/Case Management 
External Database Interfaces 


Effect 3: Administrative Productivity Gain Due to Improved Verification 
Metric: Total Number of Administrative FTEs 












TP 

Number of OW Duration of OW 

\pplication to be Verification 
Verified process per App. 


Product 


Number of Walk- 
in Application to 
be Verified 






BTP 


Number of ODSP eee of 
Application to be Secneine 


Verified 





process per App. 


Product 


Duration of Walk- 
in Verification 
process per App. 





Product 
Sum 


Total Time to 
Conduct OW 
Verification 
(minutes/year) 


Total Time to 
Productive 


Conduct ODSP 
Verification 
(minutes/year) 


Minutes per FTE 





Divide by Divide by 


Clerical and IMO Product 
Gross Up, OW and 


ODSP 









Total Number of 
FTEs for 
Verification 


Net: Effect due to BTP 


Level 3A 


Reference: 1.1.1 Social Assistance Blueprint - Implementation Plan and Business Case 


Functional/Process Group: 
BIP Integrated Sub-Initiative: 


Effect 4: 


Metric: Number of Administrative FTEs 













Number of 
Letters per case, 
OW and ODSP 


Time to produce 


Letters, OW and 
ODSP 
(Minutes/ Letter) 






Product 





Time to Produce 

Letters, OW and 
ODSP 

(Minutes/Case) 





Sum 






| 
Time to Produce 


Total Number of 






Cases, OW and Letters and 
ODSP Forms, OW and 
(Number/year) ODSP 
(Minutes/Case) 








Product 













Total Time to 
Produce Letters 
and Forms, OW 
and ODSP 
(Minutes/year) 







Divide by 


IMO Gross Up, Product 


OW and ODSP 





Base 







Net: Effect due to BTP Total Number of 
FTEs to produce 
Letters and 


Forms 






Level 3A 








Case Management 
Case Management System 


Administrative Productivity Gain Due to Automated Letters and Forms 


Number of Forms 


Time to produce 
per case, OW 
and ODSP 


forms, OW and 
ODSP 
(Minutes/ Form) 





Product 











Time to Produce 
Forms, OW and 
ODSP 
(Minutes/Form) 





Productive 


Minutes per FTE 





Reference: 1.2.4 Social Assistance Blueprint - Implementation Plan and Business Case 


Functional/Process Group: Intake/Case Management 
BTP Integrated Sub-Initiative: Centralized File Management and Eligibility 
and Entitlement Calculation Software 


Effect 5: Administrative Productivity Gain Due to Automated Overpayment Calculations 
Metric: Number of Administrative FTEs 


Number of Overpayment 
Calculations per year 


Time to Complete 
Overpayment Calculations 
(Minutes) 





Product 











Total Time to Complete 
Overpayment Calculations, 
(Minutes/year) 


Productive Minutes per FTE 





Divide by 


Product 


IMO Gross up, OW 







Total Number of FTEs to 
Perform Overpayment 
Calculations 


Net: Effect due to BTP 


Level 3A 


Reference: 1.3.2 Social Assistance Blueprint - Implementation Plan and Business Case 


Functional/Process Group: Case Management 
BIP Integrated Sub-Initiative: Automated Client Access to Information 


Effect 6: Administrative Productivity Gain due to Reduced Time Answering Questions 
Metric: Number of Administrative FTEs 










1 - Proportion of Calls 
answered by IVR 
(%) 







Number of Calls per year 


Call duration 
(Minutes/Call) 





Product 






Time Spent Answering 
Questions 
(Minutes) 


Productive Minutes per FTE 





Divide by 


Clerical and IMO Product 
Gross Up, OW and 


ODSP 








Net: Effect due to BTP 


Total Number of FTEs to 
Answer Questions 


Level 3A 


Reference: 1.2.1 Social Assistance Blueprint - Implementation Plan and Business Case 


Functional/Process Group: Case Management 
BTP Integrated Sub-Initiative: Automated Income Reporting System (via IVR) 


Effect 7: Administrative Productivity Gain due to Income Reporting via IVR 
Metric: Number of Administrative FTEs 


Note: As all Income 
reporting will be automated, 
BTP Administrative Time 








Administrative Time per 
Income Reporting per case 
by ODSP/OW 





Number of Cases by 
ODSP/OW 


for Income Reporting will be 
due to Client data entry 
discrepancies and 
mandatory reviews. 









Product 





Total Income Reporting 
Time 
(Minutes) 


Productive Minutes per FTE 










Divide by 


Clerical and IMO Product 
Gross Up, OW and 


ODSP 









Total Number of FTEs for 
Income Reporting 


Net: Effect due to BTP 





Level 3A 


Reference: 1.2.3 Social Assistance Blueprint - Implementation Plan and Business Case 


Intake . 
Disability Adjudication Unit 


Functional/Process Group: 
BTP Integrated Sub-Initiative: 


Effect 8: 
Metric: 


Administrative Productivity Gain due to Elimination of ODSP Financial Assessment 
Number of FTEs for Disability Adjudication Unit 







Proportion Self- 


Number of ODSP 


Applications Referred to 


ODSP 





Product 












OW Cases in Non-joint 
Intake sites requiring new 
financial assessment 










Clerical and IMO 


Gross up, ODSP 





Net: Effect due to BTP 


Reference: Disability Determination Benefits Forecast 
Monthly Details 





Total Time to Complete 
Financial Assessment of 
OW Referrals in ODSP 


Product 


Proportion 


Referred from 
OW 






Time to Complete Financial 
Assessment of OW 
Referrals in ODSP 


Product 


Productive 


Minutes per FTE 





Product 


Total Number of FTEs for 
ODSP Financial 
Assessment 


Level 3A 


Functional/Process Group: Case Management 
BTP Integrated Sub-Initiative: OW Employment Supports System Redesign 


Administrative Productivity Gain due to Organization Redesign 
Number of FTEs for OW Employment Supports 


Effect 9: 
Metric: 











Total Number of Cases, 


Proportion Requiring OW 
OW 


Employment Supports 


Product 















Total Number of Cases, 
Requiring OW Employment 
Supports 


Time per OW case for 
Employment supports 


Productive 







Total Time for OW 
Employment supports 


Minutes per FTE 





Product 
Clerical and IMO Product 


Gross up, OW 










Total Number of FTEs for 


Net: Effect due to BTP OW Employment Supports 


Level 3A 


Functional/Process Group: Intake/Case Management/Payment and Financial 
BTP Integrated Sub-lInitiative: Performance Management System 


Effect 10: Administrative Productivity Gain - Performance Management System 
Metric: Number of FTEs 









OW Productivity 





Total Number of 
Caseworkers 


Product 





Change in FTEs due to 
Performance Management 


Net: Effect due to BTP 


Level 3A 


Functional/Process Group: Intake/Case Management 
BTP Integrated Sub-Initiative: Centralized File Management and Eligibility 


and Entitlement Calculation Software 
Effect 11: | Program Savings due to Increased Recovery of Benefit Overpayments 
for Clients Transferring Programs 
Metric: Number of Overpayments Cases Captured per year 


Proportion of Proportion of Cases 


Total Number of 


1 - Benefit 
Capture Rate 
(%) 


Cases with 
Overpayments 
(%) 


Transferring within OW 
with an Overpayment 
(%) 


Cases, OW 





Product 


Proportion of 
Cases with 
Overpayments 
(%) 


Proportion of Cases 
Transferring from OW 
to ODSP with an 
Overpayment (%) 


Total Number of 
Cases, ODSP 


1 - Benefit 
Capture Rate 
(%) 





Product 


Sum 


Months of Monthly Number of Cases with 


Average Monthly 


Overpayment, 


OW Benefit Overpayment Overpayments Not 


Recovery Rate Captured, OW and 
(%) ODSP 





Product 










Net: Effect due to BTP Overpayments 
Not Recovered, 


OW and ODSP 


Level 3B 


Reference: 1.3.1 Social Assistance Blueprint - Implementation Plan and Business Case 


Functional/Process Group: Case Management 
BIP Integrated Sub-Initiative: Priority Based Assessment System 


Effect 12: | Program Savings due to a Reduction in the Acceptance Rate of 
Ineligible Cases 
Metric: Number of Ineligible Cases Accepted per year 





Ineligibility Current 






Ineligibility future 






Ineligible Cases Accepted, 
% of Total Caseload 





Total Number of OW Cases 








Product 


Net: Effect due to BTP 





Number of Ineligible Cases 
Accepted per year 





Level 3B 


Reference: 1.2.2 Social Assistance Blueprint - Implementation Plan and Business Case 


Functional/Process Group: Intake/Case Management 
BTP Integrated Sub-Initiative: External Database Interfaces 


Effect 13: | Program Savings due to Reduction in Entitlement Overpayment 
Metric: Number of Cases with Incorrect Entitlement Detected 

















Total Caseload 
(Number) 


Proportion of Cases with 
Overpayments Detected 
(%) 







Detection Rate 
(%) 






Product 







Net: Effect due to BTP Number of Cases with 
Incorrect Entitlement 


Detected 


Level 3B 


Reference: Consolidated Verification Process 


Functional/Process Group: Intake 7 
BTP Integrated Sub-linitiative: Disability Adjudication Unit 


Effect 14: Program Savings due to Disability Determination 
Metric: Number of ODSP Expenditures per year 







Number of ODSP 


Applications Grant Rate 





Product 






Average Allowance, OW 
and ODSP 


Number of Grants 





Product 






Net: Effect due to BTP 


Program Expenditure, 
ODSP Grants 


Level 3B 


Reference: Disability Determination Benefits Forecast 
Monthly Details 


Functional/Process Group: Intake/Case Management/Payment and Financial 
BTP Integrated Sub-Initiative: Performance Management System 


Effect 15: Program Savings - Performance Management System 
Metric: Number of Overall Payments 






% Change in Program 
Expenditure due to 
Performance Management 


Net: Effect due to BTP 


Level 3B 


Functional/Process Group: Intake/Case Management 
BTP Integrated Sub-Initiative: Centralized File Management and Eligibility 
and Entitlement Calculation Software 


Effect 16: Reduction in Ineligible Clients Receiving Entitlements 
Metric: Number of Correct Eligibility Decisions as a Percentage of Total Eligibility 
Decisions (%) 


| 
Rightly Accepted Rightly Rejected Madens Wrongly Rejected 


Sum Sum 

























Number of Correct 
Decisions 


Number of Incorrect 
Decisions 






Total Number of Decisions 


Divide by 


Number of Correct Eligibility 
Decisions as a Percentage 
of Total Decisions (%) 


Net: Effect due to BTP 


Level 3C 


Functional/Process Group: Case Management 


BTP Integrated Sub-Initiative: Centralized File Management and Eligibility 


and Entitlement Calculation Software 


Effect 17: | Reduction of Incorrect Records due to Centralized File Management 
Metric: Number of Payments Distributed to the Correct Address on Record as a 
Percentage of Total Cheques Distributed (%) 















Number of Payments 
Distributed with Incorrect 
Addresses 


Total Payments Distributed 
(Number) 










Divide by 






Number of Payments 
Distributed to the Correct 
Address on Record as a 
Percentage of Total 
Cheques Distributed (%) 


Net: Effect due to BTP 


Level 3C 


Functional/Process Group: Case Management . 
BTP Integrated Sub-Initiative: Centralized File Management and Eligibility 


Effect 18: 
Metric: 


Net: Effect due to BTP 


and Entitlement Calculation Software 


Reduction in the Distribution of Incorrect Entitlements 
Number of Payment Cheques Written in the Correct Amount as a Percentage 


of Total Cheques Written (%) 


) 


Number of Underpayments Number of Overpayments 
















Sum 





Total Number of Incorrect 
Payments 






Minus 
Total Number of Payments 








BTP 
Number of Payment 
Cheques Written in the 

Correct Amount as a 
Percentage of Total 
Cheques Written (%) 







Level 3C 


Functional/Process Group: Payment/Financial 
BTP Integrated Sub-lInitiative: Payment Financial System 


Effect 19: Reduction in Late Cheques 
Metric: Number of on Time Cheques as a Percentage of Total Cheques Distributed (%) 














Number of Payment 
Cheques Distributed on 
Time 








Number of Payment 
Cheques Distributed Late 


Total Number of Payment 
Cheques Distributed 


Divide by 


Net: Effect due to BTP 







Number of on Time 
Cheques as a Percentage 
of Total Cheques 
Distributed (%) 


Level 3C 
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ANNEX C-II: RISK ANALYSIS WORKSHOP 
PARTICIPANTS 





PARTICIPANTS OF THE JANUARY 187 AND 28™ RISK ANALYSIS WORKSHOPS 
EA EE lla ir a ed ae seins Serer a bei ictctcero sion 


MINISTRY OF COMMUNITY AND SOCIAL SERVICES 


Kevin Costante 
Bonnie Ewart 
Angela Forest 
Allan Gunn 
Sue Herbert 
Allan Kirk 
Rose Langhout 
Peter Lowery 
Bruce Meyrick 
Pauline Moeller 
Mary Simpson 
Ann Szyptur 
Martin Thumm 


ANDERSEN CONSULTING 


Alden Cuddihey 
Tony Grant 

John Kaltenmark 
Dennis Karbach 
Barry Lewis 
Bruce Parent 
Richard Steinke 
Mike Wilson 


HICKLING LEWIS BROD 


Jean-Pierre Guevremont 
Ray Hession 

Charles Lanning 

Dr. David Lewis 









& 


Sd, 












’ -_ 
. a - ar 
nae Wate mene ai 
. . - 7 7 
; : 1 a ao hhh shen 
MIT DIZOO Voarea sis 

vadibbol neblia  o - 
om yaoT | 
emesiay pdot | 
douche eronat] | 
ciwe.] yumi © | 
ire] san | 
siortd bins | 
eueliy? soli i 


NOG BW! IML DIOIA 


Mivnevaue? vyesfTneet 
noah vad 

Suc] eptrsd’) 

arverad Dive] ff 





ANNEX C-III: BUSINESS CASE MODEL PROJECTIONS 
AND RISK ASSESSMENTS 
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1. BUSINESS CASE ANALYSIS 
| 
1.1 Purpose of the Guide 


The first step in any CPP process is the development of a Business Case Analysis. Essentially, a 
business case is a detailed investment proposal that includes an analysis of all relevant costs, 
benefits, and risks associated with a particular IT investment, along with that of its reasonable 
alternatives. This step in the CPP process is fundamental in that it forms a basis for what 
follows, i.e., RFP development, vendor selection and evaluation, and contract execution. The 
business case will serve as a useful tool to help guide the subsequent steps in the procurement 
process. 


The purpose of this guide, then, is to provide practical guidance to OPS IT and responsibility 
centre (RC) managers on how to evaluate the economic merits of alternative IT investment 
proposals using business case logic. This guide offers a step-by-step blueprint that managers can 
use to build the Business Cases needed to make informed investment decisions. It is intended to 
be adapted by managers for all relevant IT investment decisions. 


Assembling a Business Case is a collaborative effort involving all the relevant stakeholders who 
are affected by the outcome of the project or will be involved in its delivery. These 
representatives should include business specialists with an understanding of the business 
requirements to be met and IT specialists with an understanding of the costs and risks inherent in 
the technologies being considered. ! 


1.2 Organization of the Chapter 


Each section of this chapter deals with a different aspect of the IT Business Case: 


e Section 1.3, Problem or Opportunity Statement—This is a statement describing the 
circumstances leading to the development of the investment project, i.e., the questions to be 
resolved by the business case analysis and the boundaries of the investigation. It includes a 
general description of project goals and objectives that follow directly from and relate 


directly to the problem to be resolved or opportunity to be exploited (which is explicitly 
stated). 


e Section 1.4, Identification of Business Case Criteria—These are criteria which define 
outcomes or results that will be used to guide investment decision, i.e., Net Present Value, 
Benefit Cost Ratio, Pay-Back Period, etc. 





' This guide satisfies and is consistent with the Business Case preparation guidelines specified by the Management 
Board in Creating and Using a Business Case for CPP Projects, October 1998, MBS. 
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Section 1.5, Identification of Decision Environment—The decision environment consists of 
all investment options which define possibilities. 


Section 1.6, Identification of Structure and Logic of the Decision Environment—The 
structure and logic of the decision environment consist of models in structure and logic 
framework which identify all of the input variables (costs, benefits, etc.) and relationships 
that determine business case criterion. 


Section 1.7, Quantification of Model Inputs and Risk Analysis— Sources of risk and 
uncertainty are used as a basis for quantification of model inputs. This involves the 
identification of central estimates and ranges (10% upper limit and 10% lower limit) for all 
input variables. 


Section 1.8, Analysis of Results—tThe final result is a forecast of each business case criterion 
and a quantification of the probability that the forecast will be achieved. In addition, 
estimates of the probability of achieving alternative forecasts, given uncertainty in the 
underlying assumptions is given for each investment option. These results are interpreted in 
light of the problem or opportunity statement. 


Section 1.9, Summary—This is a summary of different aspects of a Business Case Analysis. 


1.3 Problem or Opportunity Statement 


The first step in the development of a business case is the formulation of a problem or 
opportunity statement. A problem or opportunity statement describes the circumstances leading 
to the development of the investment project, i.e., the questions to be resolved by the business 
case analysis and the boundaries of the investigation. 


The problem or opportunity statement should include: 


e A description of the problem to be resolved or opportunity to be exploited: Note that the 


circumstances leading to the consideration of an investment project may vary. Some 
projects may be initiated by a problem or potential need. For example, particular software 
being used to perform some function may be scheduled to be orphaned or the current 
network set-up for operations of a Ministry may fail to meet the required needs of users. 
There might also be cases where the initiation of an investment project will arise from the 
identification of an opportunity. For example, automation of a Ministry’s data management 
process or upgrade of infrastructure, i.e., desktops, LANs, backbones and WANs, may yield 
large productivity or efficiency gains. The statement of the problem or opportunity should be 
as specific as possible in identifying the problem or opportunity. The more specific the 
statement, the more helpful it will be in guiding the determination of investment options. In 
addition, care should be taken to ensure that the problem or opportunity statement is directed 
at the underlying cause of the problem, not at symptoms or effects. 
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e A statement describing how the proposed investment relates to Ministry goals and 
objectives, strategies, architectures and infrastructures: The Ministry’s goals and 
objectives, strategies, architectures and infrastructures should be defined in the Ministry’s 
Business Plan and Long Term Capital Plan. In addition, most Ministries have an overall 
information management strategy and an Jnformation Management Plan which defines 
overall IT direction for the Ministry. Consult these plans for help in relating the project’s 
goals and objectives (i.e., benefits etc.) with that of the Ministry. 


e A description of the goals and objectives of the investment project: The client Ministry must 
clearly define the project’s goals and objectives. This includes a brief summary of the types 
of benefits expected from the project, expected costs, and schedule considerations. Project 
goals and objectives must be aligned with and support the business directions and priorities 
of the Ministry. 


The problem or opportunity statement is key to the Business Case. It sets the tone and direction 
for the analysis that follows. It provides important information about project needs and strategies 
that must be considered during the preparation of the Business Case. 


1.4 Identification of Business Case Criteria 


Prior to evaluation, it is necessary to select the criteria to be used in measuring and comparing 
alternative investment options. There are a variety of such criteria available. All focus on 
incremental benefits and costs with reference to the base case option. The most common 
Business Case criteria are: 


e Net Present Value (NPV)—NPYV is defined as the present-day value of benefits minus 
the present-day value of costs associated with a scenario option. A scenario option with 
an NPV that exceeds zero means that the scenario option is economically efficient. The 
preferred option, among the alternatives, would be the option with the largest positive 
NPV. Any option with a negative NPV should be rejected. 


e Benefit-Cost Ratio—The benefit-cost ratio is defined as the present value of benefits 
divided by the present value of costs. The benefit-cost ratio indicates dollars of benefit 
per dollar of cost. Using this criterion, an option is attractive or worthwhile if the ratio is 
greater than 1.0. The preferred option, among the alternatives, would be the option with 
the highest ratio. The major drawback of this criterion is that the option with the highest 
ratio may not have the greatest payoff in aggregate terms (i.e., largest NPV). Another 
weakness is the possibility of inconsistencies in the treatment of negative effects. Some 
may be handles as reductions in benefits, while others may be considered as part of the 
costs. The benefit-cost ratio would, of course, be different, depending on whether the 
negative effects are included in the numerator (i.e., as a reduction in benefits) or in the 
denominator (i.e., as part of the costs) of the ratio. 
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e Internal Rate of Return (IRR)—IRR is defined as the discount rate at which NPV=0. 
Rate of return should exceed the pre-set hurdle rate to qualify for consideration. A 
problem arises with the IRR criterion when the net benefit stream switched “signs” (2.e., 
stream changes from positive to negative) during the analytical period. This situation 
may occur when a scenario option involves large replacement costs during the period. 
The result is more than one IRR figure. 


e Hurdle Rate—The hurdle rate is defined as a minimum threshold rate of return required 
for consideration of an investment project. A project with a rate of return below the 
hurdle rate should not be considered. 


e Pay-Back Period—The pay-back period is defined as the number of years until capital is 
recouped through the flow of benefits. A short pay-back period means less risk. This 
criterion, however, may not point to the best economic solution, because it ignores net 
benefits beyond the pay-back period. The shortest pay-back period may not have the 
highest payoff in the long term. 


e Nth-Year Benefit— Nth-year benefit is defined as benefits in the Nth-year after 
construction (design-build), divided by costs to the Nth date including interest paid 
during construction (design-build), expressed as a percentage. A ratio equal to the hurdle 
rate means the project is optimally timed. A ratio below the hurdle rate means the project 
is premature. A ratio above the hurdle rate means the project is overdue. This criterion, 
as well, may not point to the best economic solution, because it relates only to timing and 
not to the aggregate net benefits over the life-cycle of the investment. 


e Change in Person Years—The change in person years measures the reduction in the 
number of full-time equivalents (FTEs) needed to accomplish Ministry functions. This 
reduction is that which would result from implementation of the investment project. 
Using this criterion, the preferred option, among the alternatives, is the one with the 
largest reduction. 


e Efficiency of Capital Employed—As an alternative to NPV, the efficiency of capital 
utilization may be measured by dividing NPV by the total initial capital expenditure to 
obtain the net benefit per dollar of investment. This measure is more effective in 
situations where capital is relatively limited and must be applied most efficiently. 


Which Business Case criteria a Ministry chooses depends upon the goals, objectives, and 
priorities of the particular Ministry. The Ministry should look carefully at each criterion and 
evaluate which is most consistent with its needs and circumstances. The most common criterion, 
however is NPV. This is attractive in that it best measures the overall value of an investment 
over its life-cycle. In addition, the other criteria are easily calculated once NPV is given. This is 
because the inputs to the NPV model are very similar to those necessary to calculate the other 
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criteria. Therefore, we suggest that in addition to whatever other criteria a Ministry chooses, it 
include at a minimum—NPYV . 


1.5 Identification of Decision Environment 


Essentially, the decision environment consists of all investment options which define 
possibilities. 


The following questions will assist in the identification of investment options: 


e Can the problem or opportunity be addressed (solved or exploited) to different degrees? 
This question deals with matters of scope, i.e., the magnitude or extent to which an option 
accomplishes the objectives as stated in the Problem or Opportunity Statement. How well an 
option meets the objectives is something to be considered. It need not be the case that the 
Ministry fully satisfy the long-term goals and objectives with any one particular investment 
project. Note that differences in scope normally imply variations in the level of required 
investment. 


e Are there different ways of addressing (solving or exploiting) the problem or opportunity? 
Whereas the previous question dealt with how much action, this question concerns what kind 
of action can be taken. Typically, there are many different ways to solve a problem or exploit 
an opportunity. The number of IT solutions for any given objective or goal may be many, 
each with different costs, benefits, and risks. Business cases often fail when managers fail to 
look broadly at available options. 


The number and type of investment options differ, depending on the details of the investment 
project. In the end, however, the costs, benefits, and risks of each option will be compared to 
that of a base case option. A base case provides the common point of reference against which the 
costs, benefits, and risks of other options are measured. 


Although the base case is likely to resemble the status quo to a substantial degree, important 
distinctions do exist. 


e Status Quo Option—The status quo assumes that the Ministry or system will remain static, 
i.e., no change to current processes, efficiencies, technology, training, or productivity. The 
status quo option unrealistically assumes the organization is frozen in time. 


e Base Case Option—The base case assumes that there is continuous quality improvement in 
current operations, and that optimization of current resources will be realized. It specifies the 
changes to the status quo that are possible/desirable in order to improve performance within 
the current mandate and resources. 


Management Board Guidelines caution against the use of base line (status quo) assumptions that 
lead to exaggerated claims of benefits and understatements of costs. This occurs when 
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Management Board Submissions assume that in the absence of the proposed amendment, nothing 
will be done to sustain or improve performance and that no capital outlays will occur to improve 
systems and operations. A “do nothing” option is regarded as unrealistic and unacceptable as a 
basis for comparing the costs, benefits, and savings associated with other options. 


Note that many IT options may be implemented using a variety of strategies. Including 
alternative strategies increases the effective range of options available to the government. For 
example, an IT application requirement may be met by one or more of the following options: 


e Redefining business processes to achieve the desired result without making an IT 
investment; 


e Re-using or adapting an application developed by another business unit or Ministry; 

e Re-engineering the existing system (if there is one) to provide the functionality required; 
e Acquiring a commercial off-the-shelf (COTS) product; or 

e Custom building a new application. 


Strategies for building, adapting, or re-engineering an application include doing it in-house or 
contracting out, and implementing in phases or all at once. 


Options for acquiring IT hardware include: 
e Buying it; 
e Leasing it; 
e Renting it; and 
e Greening it (leasing with regular upgrades). 


Strategies for implementation include various timing profiles, i.e., delaying the investment until 
better technology is available or until the proposed technology is more widely used. 


1.6 Identification Of Structure And Logic of the Decision Environment 


The structure and logic of the decision environment consists of a detailed description in structure 
and logic framework of the expected benefits, costs, and relationships of various IT investment 
options which determine Business Case criteria. 


Associated with each investment option, are costs and benefits that have a strong impact on an 
investment’s overall value. Therefore, close attention should be given to completely identifying 
all relevant costs and benefits associated with each option over the life of the project. 
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1.6.1 Identifying Cost Inputs 


Sound investment decisions necessarily include a complete evaluation of all costs associated 
with the investment. Therefore, the business case should reflect the full cost of the investment 
project, including initial up-front costs, ongoing costs, and other indirect costs over its life-cycle. 
Costs associated with IT projects may be broken down into the following categories: 


e Design Costs 

e Building Costs 

e Implementation Costs 

e Operating Costs 

e Maintenance Costs 
Costs within each of the above categories typically are of one of three types: 

e Capital Costs 

e Labour Costs 

e Out-Source Contract Costs 
Below is a list of direct up-front (non-recurring) and ongoing (recurring) costs typically 
associated with IT investment projects. If relevant, these costs should be included in the above 
categories: 

¢ programming costs 

e cost of hardware and peripherals 

e software (packaged and customized) costs 

e cost of initial data collection or conversion of archival data 

¢ facilities upgrade (including site preparation and renovation) costs 

e cost of telecommunications equipment 

e user specification costs 

e cost of initial user training 


e workforce adjustment costs (for affected employees) 


HICKLING LEWIS BROD INC. PAGE a 


transition costs (such as costs of running parallel systems or converting legacy systems) 
salary and benefit costs of IT staff 

out-source contract costs 

software maintenance and upgrade costs 

cost of computer supplies 

user support costs 

cost of ongoing training 


cost of quality assurance and post-implementation reviews 


In addition to these costs, IT investment projects may have the following other costs: 


Indirect (Hidden) Costs: Typically, these costs are caused by an initial loss of 
productivity. (At first, productivity may drop while users learn new IT tools). In some 
cases, hidden costs may even exceed direct costs. For example, it has been reported that 
hidden on-going costs of local and wide area networks can be three times that of direct 
on-going cots (Managing End User Computing, Nolan, Norton, & Co., 1992.) 


Client Costs: Government IT projects may increase costs to clients (i.e., when 
government costs are shifted to the client). Client costs may be direct or indirect and are 
similar to those faced by the Ministry. 


Enterprise-Wide Costs: IT investments may affect individuals and groups that are neither 
users nor IT service providers (i.e., investment projects often change work processes, 
sometimes altering how work is performed across the Ministry). 


1.6.2 Identifying Benefit Inputs 


The range of benefits derived from various IT investment projects differs depending on the 
project under consideration. Typically, however, benefits fall into one of three categories: 


Productivity (Efficiency) Benefits 
Technological Benefits 


Client (Customer) Service Benefits 
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Productivity (Efficiency) Benefits 


Productivity or efficiency benefits are typically in the form of non-IT labour cost savings. Non- 


IT labour cost savings may result from one or more of the following: 
e elimination of duplication or redundancy of data and process 
¢ increased number of transactions and functions performed electronically 
e increased speed of performing transactions and functions 
e improved workflow and process 


e increased capacity 


In addition, IT investments may yield other productivity or efficiency benefits that relate to: 


e new opportunities from ease of access and use of information 
e better-informed decisions 


Technological Benefits 


Technological benefits also must be considered. Technological benefits usually relate to IT 


reliability and maintainability. 


Reliability benefits might include: 


e Increase in Mean Time Between Failure (MTBF): MTBF is defined as the arithmetical 
average of operating time between system equipment malfunctions when the equipment 


are operated in accordance with the duty cycles defined. 


e Increase in Mean Time Between Computer Program Events (MTBCPE): MTBCPE is 
defined as the arithmetical average of operating time between system major/critical 
events during a 72-hour period when the system elements are operated in accordance with 
the duty cycle defined. Major/critical malfunctions events are those malfunction events 


which reduce system performance below the performance level as defined. 


e Increase in Mean Time Between Critical Failure (MTBCF): MTBCF is defined as the 
arithmetical average of operating time between system critical failures when the system 


elements operated as defined. 
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e Decrease in Number of Faults: A fault is defined as an equipment malfunction that causes 
any parameter of the equipment performance to deviate from its specified limits. 


e Decrease in False Alarms: A false alarm is defined as an event in which fully operational 
equipment tested out as malfunctioning. 


e Decrease in Number of Missed Detections: A missed detection is defined as an event in 
which malfunctioning operational equipment tested out as fully functional. 


e Decrease in Number of Critical Computer Program Errors: A critical computer program 
error is defined as any error for which the computer programs are unable to support 
normal system operational functions. 


Maintainability benefits might include: 


e Decrease in Mean Time To Restore (MTTR): MTTR is defined as the arithmetical 
average of the time it takes to restore computer system operation following computer 
system interrupts caused by computer equipment or program events. 


e Decrease in Mean Logistics Down Time (MLDT): MLDT is defined as the arithmetical 
average of the logistic delay time associated with system failures that degrade 
performance below the mission reliability performance levels and measured over an 
operating cycle. Logistic delay time is measured for all malfunctions that degrade 
performance below mission reliability performance criteria that cannot be corrected with 
onboard resources. Causes of logistic delays include: spare or repair parts, 
documentation, test equipment or skilled technicians not onboard. Logistic delay time for 
system failure corrected with onboard spares, equipment and personnel is zero. 


Client (Customer) Service Benefits 
Note that the government Ministry investing in the IT project is not the only one who may 
benefit. Often, the Ministry’s clients or beneficiaries benefit as well. These benefits might 
include the following: 

e less paperwork (for clients) 


e less time spent getting information and services 


¢ improved service (increased quality and/or quantity of service) 


1.6.3 Discount Rate Variable 


Future IT costs and benefits must be converted to present values so that they can be compared on 
a common basis. Discounting provides the means to do this. The need to do discount stems 
from the fact that, even when dealing with constant dollars, the value that is placed on costs and 
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benefits depends on when they occur. A dollar to be received a year from now is worth less than 
the dollar in one’s pocket today, because of opportunities foregone during the year. Note that it 
is preferable to accrue benefits earlier rather than later and to delay expenditures. 


The present value (PV) of a future cost or benefit is determined by the formula: 


PV Gees Clr)” 
where: 


s = future value 
r = the annual discount rate 
n = number of years from the base year. 


The difference between a future value and its corresponding present value increases with the 
number of intervening years and the discount rate. The present value of a stream of future values 
is the sum of the present values of each element of the stream. 


Although any base year may be selected, the base year for discounting is usually the same as the 
year in which the decision is made. 


1.6.4 Calendar or Fiscal Year 


Dollars may be expressed on either a fiscal year or a calendar year basis, but not mixed in the 
same analysis. In many cases, it is convenient to adopt a calendar year basis, because of the 
greater availability of calendar year statistics and forecasts. For project evaluations involving 
benefits or costs that accrue on a fiscal year basis, such as government labour cost savings, it 
may be simpler to work on a fiscal year basis. Whichever basis is chosen, it must be used 
consistently throughout the analysis. 


1.6.5 Evaluation Period 


The costs and benefits of each scenario option are to be evaluated over a timeframe equivalent to 
the economic (useful) life of the investment option. When options involving assets with different 
economic lives are being compared, it is recommended that the analytical timeframe equal the 
useful life of the most durable asset. 


1.6.6 Timing Profile Variables 


Timing variables which reflect the time frame in which benefits and costs are incurred must be 
included as well. Timing variables typically relate to implementation and out-placement 
profiles. The implementation profile is usually given in terms of implementation start (months 
from project start) and duration (months from implementation start). The out-placement profile 


is usually given in terms of out-placement start (months from implementation start) and duration 
(months from first out-placement). 
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1.6.7 Structure and Logic of the Decision Environment 


Once the costs and benefits of the decision environment have been identified, it is time to 
develop “Structure and Logic Models” which reflect this methodology. A Structure and Logic 
Model is the identification of all of the variables and relationships that determine a business case 
criterion. It spans the whole of the decision environment. A Structure and Logic Model depicts 
the methodology non-mathematically, indicating how all variables and assumptions combine to 
yield a forecast. 


In developing the Structure and Logic Models, it is important to understand and identify all 
factors and relationships that make up each cost and benefit input. These factors and 
relationships must be included in the Structure and Logic Models themselves. 


Below are a few steps that will aid in the development of these Structure and Logic Models. 


e Step 1: Begin with the decision criterion of interest expressed in terms of benefits and 
costs in constant dollars. For example, (assuming the Business Case criteria is NPV) see 
Figure 1. 


e Step 2: Develop the constituent components of the benefits and costs in terms of input 
variables and their relationships. See Figures 2 and 3. 


e Step 3: Work further up the tree until variables have been broken into all relevant input 
values. Depict in Structure and Logic format, all the factors and relationships that make 
up each cost and benefit input. In other words, take each benefit and cost input (in 
Figures 2 and 3) and break them down into small enough pieces such that reliable data is 
available. For example, see Figures 4 and 5. 


Note that each of the inputs in Figure 5 may be further broken down into its component inputs. 
For example, see Figure 6. Continue to work further up the tree until each variable has been 
broken into all relevant factors and relationships (input variables) that compose it. This should 
be done until there exists sufficient reliable data for each input variable in question. 


Note that the exact specification of each cost and benefit input will depend upon the problem or 
opportunity being considered. Figure 5, for example, is merely one possible specification for 
how capital costs are determined. It should be the case, however, that the overall model as well 
as the various input models is general enough to be used for each investment option. Begin with 
the status quo, and then add variables for the base case and alternatives as appropriate. It may be 
the case, that in some scenario options, certain inputs are not relevant, i.e., zero. 


Note also that the cost and benefit categories depicted here merely serve to help identify all the 
various costs and benefits associated with a particular IT project. It is not necessary that the 
structure and logic models reflect this particular classification. How total costs and benefits are 
allocated across cost and benefit input categories does not much matter, as long as all costs and 
benefits are accounted for. The cost input categories, though, should reflect the way in which the 
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data are available. Therefore, before allocating total costs and benefits across input categories, 
first check to see the form in which the cost data is given. Typically, this will be the way in 
which the Ministry’s budget is structured. 
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Figure 1: Net Present Value 
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Figure 2: Total Benefits 
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Figure 3: Total Costs 


Design Costs Implementation Costs Maintenance Costs 
By Year By Year By Year 
Real $/year Real $/year Real $/year 


Building Costs Operating Costs 
By Year By Year 
Real $/year Real $/year 


Total Costs 
By Year 
Real $/year 


HICKLING LEWIS BROD INC. 





PAGE 


16 


Figure 4: Building Costs 
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Figure 5: Capital Costs 
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Figure 6: Hardware & Network Costs 
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1.7 Quantification of Model Inputs and Risk Analysis 


Quantification of model inputs involves the identification of values for all input variables. It is 
important that every cost and benefit associated with each investment option be quantified. In 
most cases, this can be done without much trouble. There are however, instances where 
quantification is more difficult. Typically, these difficulties arise on the benefit side (quantifying 
costs is usually fairly straight forward). In these cases, there are some well-tested methods that 
may prove helpful in determining approximate value of hard-to-quantify benefits. They are 
given below: 


e Quantify the value of the benefit’s effects. 
e Set value equal to the cost of alternative solutions. 
e Set value equal to the cost of not providing the benefit. 


Note that MBS Business Case preparation guidelines require that both tangible and intangible 
benefits are quantified. Though each benefit may not reduce to financial cash-flow effects, they 
do reduce to economic effects. 


Because costs and benefits extend into the future (over the life-cycle of the investment), forecasts 
of model variables necessarily involve risk and uncertainty. This risk and uncertainty must be 
accounted for in the development of the values given to the model inputs. A proper approach to 
quantifying model input values in light of uncertainty involves the use of Risk Analysis as a 
foundation by which the planner can evaluate alternative options under a variety of scenarios. 


1.7.1 Risk Analysis 


The result of a risk analysis is both a forecast and a quantification of the probability that the 
forecast will be achieved. Not unlike modern weather forecasting, in which the likelihood of rain 
is projected with a statement of probability ("there is a 20 percent chance of rain tomorrow"), 
Risk Analysis is intended to provide the Ministry with a sense of perspective on the likelihood of 
future events. Risk Analysis is an easily understandable, but technically robust method that 
allows planners and decision-makers to select the level of risk within which they are willing to 
plan and make commitments. 


The further into the future projections are made, the more uncertainty there is and the greater the 
risk is of producing forecasts that deviate from actual outcomes. Projections need to be made 
with a range of input values to allow for this uncertainty and for the probability that alternative 
economic, demographic, and technological conditions may prevail. The difficulty lies in 
choosing which combinations of input values to use in computing forecasts, and how to use those 
forecasts to produce a final estimate. 


Forecasts traditionally take one of two forms: first, a single "expected outcome", or second, one 
in which the expected outcome is supplemented by alternative scenarios, often termed "high" and 
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"low" cases. Both approaches fail to provide adequate perspective with regard to probable versus 
improbable outcomes. 


The limitation of a forecast with a single expected outcome is clear -- while it may provide the 
single best guess, it offers no information about the range of probable outcomes. The problem 
becomes acute when uncertainty Surrounding the underlying assumptions of the forecast is 
especially high. 


The high case-low case approach can actually exacerbate this problem because it gives no 
indication of how likely it is that the high and low cases will actually materialize. Indeed, the 
high case usually assumes that most underlying assumptions deviate in the same direction from 
their expected value; and likewise for the low case. In reality, the likelihood that all underlying 
factors shift in the same direction simultaneously is just as remote as everything turning out as 
expected. 


A common approach to providing added perspective on reality is through "sensitivity analysis", 
whereby key forecast assumptions are varied one at a time in order to assess their relative impact 
on the expected outcome. A problem here is that the assumptions are often varied by arbitrary 
amounts. But a more serious flaw in this approach is that in the real world, assumptions do not 
veer from actual outcomes one at a time; it is the impact of simultaneous differences between 
assumptions and actual outcomes that would provide true perspective on a forecast. 


Risk Analysis provides a way around the problems outlined above. It helps avoid the lack of 
perspective in "high" and "low" cases by measuring the probability or "odds" that an outcome 
will actually materialize. This is accomplished by attaching ranges (probability distributions) to 
the forecasts of each input variable. The approach allows all inputs to be varied simultaneously 
within their distributions, thus avoiding the problems inherent in conventional sensitivity 
analysis. The approach also recognizes interrelationships between variables and their associated 
probability distributions. 


To incorporate Risk Analysis into the development of initial values of model inputs: 


e First, integrate the Structure and Logic Model developed earlier into risk analysis 
software. 


e Then, for each input variable, enter into risk analysis software, a central estimate and a 
range (10% upper limit and 10% lower limit) which represents the degree of uncertainty. 


Note that in addition to incorporating Risk Analysis into the forecasts of model inputs, other risk 
monitoring activities must be performed. To satisfy MBS requirements, a capability evaluation 
of the client Ministry, a risk assessment of the investment project and a risk management plan 
must also be prepared. These activities are discussed in other chapters of this guide. 
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1.7.2 Data Collection 


Prior to quantification of model inputs, it is important for client Ministriess to gather certain 
kinds of data. Doing this will facilitate the quantification process. Ministriess should collect the 
following types of data: 


e Data on the user requirements of the Ministry. User requirements are typically expressed 
in terms of user demand. This includes a description of the number and type of users 
affected by the investment project. 


e Data on the existing technical and infrastructure conditions affecting the investment 
project. This includes a description of the current situation, i.e., description of relevant 
hardware, network, and software conditions in the Ministry. 


The quantification of central estimates and ranges for each input variable is performed using 
special data sheets. A sample data sheet is given in Figure 7 below. Data sheets and variable 
explanations should be prepared for each input variable and investment option over the time 
period of interest. 


The first column provides space for an initial median estimate, and the second and third columns 
define a range which represents "an 80 percent confidence interval" -- the range within which we 
can be 80 percent confident of finding the actual outcome. Thus the greater the uncertainty 
associated with a forecast variable, the wider the range will be (and vice versa). 
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Figure 7: Sample Data Sheet 


_ Data Sheet ~ 
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Note that it is important to distinguish between input and result variables. We advise that 
intermediate inputs be treated as result variables. If an input is broken down into other inputs, 
then it is considered an intermediate input and therefore treated as a result variable. For example, 
the Capital Costs variable is actually an intermediate input, derived from other input variables, 
i.e., Hardware & Network Costs, Software Costs, PC Costs, etc.. Therefore, the Capital Costs 
variable is treated as a result variable. 


When an input is not able to be broken down any further, it is considered an input variable. 


Only input variables are put into the Data Sheets. The input variable estimates (median, 10% 
low, 10% high) can be gleaned through a variety of means, including research, stakeholder 
interviews, industry standards, etc.. 


The probability ranges for each variable will depend upon the degree of risk and uncertainty 
inherent in each variable. Note that probability ranges for the variables in-question may be 
established on the basis of both statistical analysis and subjective probability. Ranges need not 
be normal or symmetrical -- that is, there is no need to assume the bell shaped normal probability 
curve. The bell curve assumes an equal likelihood of being too low and being too high in 
forecasting a particular value. It might well be, for example, that if projected inflation rates 
deviate from expectations, they are more likely to be higher rather than lower. It is important to 
place no restrictions on the degree of "skew" in the specified ranges and thus maximizes the 
extent to which the Risk Analysis reflects reality. 


Because the probability ranges for each variable will depend upon the degree of risk and 
uncertainty inherent in each variable, an important first step when developing these input 
variable estimates is to properly identify and evaluate all relevant risk factors which might affect 
the investment project. This allows the preparer to assess how risk and uncertainty inherent in 
the project will affect the attainment of alternative input variable values. 


1.7.3 Risk Factors 


The nature of risk in IT-related projects will differ depending on the project and investment 
option considered. However, IT risk typically falls into one of the following five categories: 


e Financial Risk 

e Economic Risk 

e Technological Risk 
e Political Risk 

e Human Risk 


Factors affecting the financial risk of an investment project include: 
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e capital cost overruns 
® scope creep 
e slippage 
e financial stability of partners and contractors 
e federal fiscal framework 
e maintenance and operating cost overruns 
Factors affecting the economic risk of an investment project include: 
e interest rate climate 
e market valuation of assets and products 
Factors affecting the technological risk of an investment project include: 
e shake out 
e human factors (with respect to the utilization of technology) 
e obsolescence 
e reliability 
Factors affecting the political risk of an investment project include: 
e political resistance to changes in responsibilities and reporting structures 
e political resistance to down-sizing 
e external political resistance to changes in the organizational mandate 
Factors affecting the human risk of an investment project include: 
e training budget shortfalls 
e labour resistance to down-sizing 
¢ individual resistance to changes in job descriptions or processes 


¢ individual resistance to changes in the organizational or reporting structure 
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Once, all the relevant risk factors affecting a proposed project are identified, they should be used 
in the identification of the ranges (10% low, 10% high) placed around each central estimate. In 
addition, any relevant information resulting from the Capability Evaluation and Risk Assessment 
should also be used to identify the uncertainty inherent in each input variable. 


1.7.4 Monte Carlo Simulation 


Once these ranges are identified, Monte Carlo simulation methods should be used to generate 
“probability density functions” for each input variable. This can be done using some type of risk 
analysis software. Risk analysis software uses numerical analysis to transform each range into 
formal "probability density functions" automatically. Note that these “probability density 
functions” do not have to be determined or presented in either mathematical or graphical form. 
All that is required is the entry of each variable’s central estimate along with its 80 percent 
confidence interval (10% low value and 10% high value) into the risk analysis software from the 
Data Sheets. This liberates the non-statistician from the need to appreciate the abstract statistical 
depiction of probability and thus enables administrators, stakeholders and decision-makers to 
understand and participate in the process whether or not they possess statistical training. 


1.8 Results and Analysis 


The “probability density functions” for each input variable should then be combined, again using 
Monte Carlo Simulation, into individual result variables which are then used to calculate the 
various business case criteria of interest. This is done automatically by the risk analysis 
software. 


The final result is a forecast of the business case criterion, together with estimates of the 
probability of achieving alternative outcomes given uncertainty in the underlying assumptions. 


This information is presented graphically (See Figure 8) and in table format (See Figure 9). 
Figure 8 expresses the value of the business case criterion for a particular option along the x-axis 
and the probability of exceeding particular values along the y-axis. A separate figure is given for 
each investment option, for each business case criterion of interest. Figure 9 expresses the value 
of the business case criterion of interest for alternative investment options, under various 
probability assumptions. A separate table is given for each business case criterion of interest. 


Figure 10 underscores the importance of developing a probability-based (risk-based) business 
case, i.e., incorporating risk analysis into the Business Case Analysis process. Though Options 
A and B possess similar expected NPVs (with a 50% probability of occurring), Option B entails 
significantly more risk. A (non-probability-based) traditional business case would only report 
the value with a 50% probability of occurring. 


Once the results are presented, a careful analysis of investment options must be performed. This 


analysis allows the government to rank alternative investment options according to business case 
criterion. 
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Note that it is important to compare options from a probability-based (risk-based) perspective as 
opposed to simply comparing expected-values (outcomes with a 50% probability of occurring). 
The effect of risk on the Business Case results is often quite dramatic and provides key 
information to the decision-maker. 


In many instances, the ranking of alternative investment options according to business case 
criteria, changes, depending on the probability level of interest. For example, consider the results 
in Figure 10. When considering outcomes with a 50% probability of occurring, Option B would 
be ranked above Option A. However, when considering outcomes with a 90% probability of 
occurring, Option A would be ranked above Option B. 


MBS recommends a multi-criteria approach to options selection and analysis. As part of the 
selection and analysis process, the following tasks must be performed: 


e For all business case criteria, decide what level of surety (expressed as probability of 
obtaining outcome values) the government is comfortable with for making selection 
decisions, i.e., outcomes with a 50% probability of occurring, 80% probability of 
occurring, or 90% probability of occurring, etc.. 


e Rank (via a weighting scheme) alternative business case criteria in order of importance to 
the Ministry. 


e Rank each investment option within each business case criterion. 


See Assessing the Value of Information Technology, NCR Corporation, for various models used 
to compare alternative investment options. Note that these models each use as their foundation 
the results of the business case analysis, i.e., the outcome values associated with each business 
case criteria. 
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Figure 8: Final Result (Graphical Format) 
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Figure 9: Final Result (Table Format) 


Business Case Criterion (i.e., NPV)* 


Business Case Criterion 
(50% Probability of Exceeding) 
Business Case Criterion 
(50% Probability of Exceeding) 
Business Case Criterion 
(80% Probability of Exceeding) 
Business Case Criterion 
(90% Probability of Exceeding) 


Business Case Criterion 
(95% Probability of Exceeding) 


*Discount Rate: 


*Evaluation Period: 
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Figure 10: Comparison of Options 
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1.9 Summary 


This section is intended to provide practical guidance to Ministries on the methods and process 
of evaluating the economic merits of alternative investment proposals, using business case logic. 
This guide offers a step-by-step blueprint for developing business case analyses which allow 
managers to make informed investment decisions. It is intended to be adapted by managers for 
all relevant investment decisions. 


Essentially, a business case analysis is a detailed investment proposal which includes an analysis 
of all relevant costs, benefits, and risks associated with a particular investment project, along 
with that of its reasonable alternatives. This step in the Common Purpose Procurement(CPP) 
process is fundamental in that it forms a basis for each of the other steps of the process, i.e., RFP 
development, vendor selection and evaluation, and contract execution. 


Below is a summary of each step of the business case development process. 


e Write a Problem or Opportunity Statement—This is a statement describing the 
circumstances leading to the development of the investment project, i.e., the questions to be 
resolved by the business case analysis and the boundaries of the investigation. It includes a 
general description of project goals and objectives that follow directly from and relate 
directly to the problem to be resolved or opportunity to be exploited (which is explicitly 
stated). 


e Identify Business Case Criteria—These are criteria which define outcomes or results that 
will be used to guide investment decision, i.e., Net Present Value, Benefit Cost Ratio, Pay- 
Back Period, etc. 


e Identify Decision Environment—The decision environment consists of all investment 
options that define possibilities. 


e Identify Structure and Logic of the Decision Environment—The structure and logic of the 
decision environment consist of models in structure and logic framework which identify all 
of the input variables (costs, benefits, etc.) and relationships that determine business case 
criterion. 


e Quantify Model Inputs Using Risk Analysis— Sources of risk and uncertainty are used as a 
basis for quantification of model inputs. This involves the identification of central estimates 
and ranges (10% upper limit and 10% lower limit) for all input variables. 


e Present and Analyze Final Results—The final result is a forecast of each business case 
criterion and a quantification of the probability that the forecast will be achieved. In 
addition, estimates of the probability of achieving alternative forecasts, given uncertainty in 
the underlying assumptions is given for each investment option. 
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